We Keep Coding

how can i redirect my error message from console to file in Python [duplicate]

This question already has an answer here:
redirect_stderr does not work (Python 3.5)
(1 answer)
Closed last year.
I need to redirect my error message from the console to a file. For this example, I need to insert the error message into a file:
Traceback (most recent call last):
File "C:/Users/", line 5, in <module>
1/0
ZeroDivisionError: division by zero"
I have already tried to do something like this:
from contextlib import redirect_stdout
with open('error.txt', 'w') as f:
with redirect_stdout(f):
1/0
print('here is my error')

If you plan to run your script in console itself, you can just use the bash's ">" operator to send the input of your command (in this situation : your script) in a file just like this :
python ./yourScript > ./outputFile
Everything that your script will print will go in the specified file.

You need to catch the error or your application will fail:
from contextlib import redirect_stdout
with open('error.txt', 'w') as f:
try:
1/0
except ZeroDivisionError as e:
f.write(e)

Note: This assumes you're using Bash. I see that you are using Windows, so it's likely that you aren't using Bash. But from what I've read, this should still be applicable if you are using Cmd.exe, it's just that the syntax might be slightly different.
I think it's better to handle error message output outside of your script. Your script should attempt to do the "happy path" work and print an error to stderr if something goes wrong. This is what should happen by default in every programming language. Python gets this right.
Here is an example script:
print("Dividing by 0 now, I sure hope this works!")
1/0
print("Holy cow, it worked!")
If I run this script, the first line prints to stdout, and then the ZeroDivisionError output prints to stderr:
$ python /tmp/script.py
Dividing by 0 now, I sure hope this works!
Traceback (most recent call last):
File "/tmp/script.py", line 3, in <module>
1/0
ZeroDivisionError: integer division or modulo by zero
If I want to run the script and collect any error output in a file, I can use redirection in my shell when I run the command:
$ python /tmp/script.py 2> /tmp/errors.txt
Dividing by 0 now, I sure hope this works!
$ cat /tmp/errors.txt
Traceback (most recent call last):
File "/tmp/script.py", line 3, in <module>
1/0
ZeroDivisionError: integer division or modulo by zero

Can I modify what pdb prints when an Exception is thrown?

I run:
python -m pdb script_that_throws.py
When the script throws, say because of a failed assertion, pdb prints the entire stack trace + some pointless boilerplate text:
Traceback (most recent call last):
File "/usr/lib/python3.6/pdb.py", line 1667, in main
pdb._runscript(mainpyfile)
File "/usr/lib/python3.6/pdb.py", line 1548, in _runscript
self.run(statement)
File "/usr/lib/python3.6/bdb.py", line 434, in run
exec(cmd, globals, locals)
File "<string>", line 1, in <module>
[... many lines of stack trace omitted for brevity ...]
File "/path/to/script_that_throws.py", line 26, in _ul
assert v.keys() == self._expected_keys
AssertionError
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> /path/to/script_that_throws.py(26)_ul()
-> assert v.keys() == self._expected_keys
(Pdb)
I would like for Python to only show:
AssertionError
> /path/to/script_that_throws.py(26)_ul()
-> assert v.keys() == self._expected_keys
(Pdb)
Is there a way to achieve that?

There is a good and a bad answer.
The good one. Certainly, you can suppress these messages.
And now the bad answer comes into play.
You have to change cpython source code, and recompile it.
This is not so hard as it sounds at first, especially, as pdb is a Python module, no C knowledge required.
This is the important part of the source code:
https://github.com/python/cpython/blob/master/Lib/pdb.py#L1718-L1725
There is excellent documentation out there on how to compile Python:
https://docs.python.org/3/using/unix.html#building-python
Also, Anthony Shaw just published a book which gives a good introduction on how to work with cpython, the C implementation of the Python language.
Just in case you expected a different kind of answer, no, there is no configuration option for PDB to suppress these kind of messages.

It would be nice to know what script_that_throws.py does. Have you tried using try and except in your script? This can help you handle the pdb exceptions to print whatever you want. Like so:
n = input("age ")
try:
print(int(n))
except ValueError:
print("error")
You can also manually raise your own exception as shown in this thread:
Manually raising (throwing) an exception in Python

How do I print Python errors to a txt file, if I don't know where the error is

I have a very large Python 3.x program running in Windows. It works great 99.9% of the time, but occasionally it crashes. I'm not sure what is causing the crash, it could be numerous things. Due to the fact that I have to run the program "compiled" .exe with an invisible console for security reasons (don't ask), I don't get to see any form of console readout when it dies. So obviously it would be great if I could have it output the crash traceback as a text file instead.
I'm familiar with try/except in Python but the piece of code that's causing the issue could be anywhere and I don't want to have to write an individual try/except around every single line of the literally thousands of lines of code. Is there a way that I can get the program to always output any program-stopping error as a text file, no matter what line of code is causing the problem, or what the error might be?

Somewhere in your code you must have a single entry-point into the code that might be crashing (in the main script, at a minimum). You can wrap that in a try/except pair and then use functions from the traceback module to print the exception to a file when it happens:
Change:
if __name__ == "__main__":
do_stuff()
To:
import traceback
if __name__ == "__main__":
try:
do_stuff()
except:
with open("exceptions.log", "a") as logfile:
traceback.print_exc(file=logfile)
raise
If you want to, you could add some extra code to write extra output to the file, with a time/date stamp or whatever other information you think might be useful. You may want to add additional try/except blocks, more or less like the one above, if you want to give special scrutiny to certain parts of your code. For instance, you could put a block in a loop, where you can print out the loop value if an exception occurs:
for x in some_iterable:
try:
do_something_with(x)
except:
with open("exceptions.log", "a") as logfile:
print("Got an exception while handling {!r} in the loop:".format(x)
traceback.print_exc(file=logfile)
raise # you could omit this line to suppress the exception and keep going in the loop
You could also use the logging module, if you want a more configurable system for the file writing end of the issue. The logging.debug and logging.exception functions both read the same exception information used by the traceback module, but with many more options for formatting things yourself (if you want that). Note that setting up logging is a bit more involved than just opening a file manually.

sometimes you cant use try/except or > in terminal.
you can use sys excepthook.
add this to beginning:
import sys
import traceback
def excepthook(exctype, value, tb):
with open("mylog.txt", "w") as mylog:
traceback.print_exception(exctype, value, tb, file=mylog)
sys.excepthook = excepthook
##########
# your code
after that, all traceback will be print to mylog.txt.

I ended up writing my own logging function
# Takes two inputs - logfile (path to desired .csv), and data to be written
# Writes "Y-M-D|H:M:S, data\n"
f = open(logfile, 'a+')
currentdate = time.strftime('%Y-%m-%d|%H:%M:%S')
f.write(currentdate + ',' + data +'\n')
f.close()
it requires time or datetime, I'm not sure which. Also you need to make sure that the log file exists.
Then I would just plop it wherever I needed, eg: logger(ERRLOG, "OCR didn't find poop. Check {}".format(ocr_outfilepath))

I'm not sure what kind of program this is or how you are running it, but you could try running your Python program and redirecting all its output (or all errors) to a file.
For example, if I have a very-contrived sample Python script like this
def do_stuff():
s = [1, 2, 3, 4, 5]
print(s[6])
if __name__ == "__main__":
do_stuff()
which is deliberately going to raise an IndexError exception.
You could run it like this:
$ python test.py &> mylogs.txt
$ cat mylogs.txt
Traceback (most recent call last):
File "test.py", line 8, in <module>
do_stuff()
File "test.py", line 4, in do_stuff
print(s[6])
IndexError: list index out of range
which redirects all output and errors to a file.
Or, if you can have it displayed on a console and also redirect it to a file:
$ python test.py 2>&1 | tee mylogs.txt
Traceback (most recent call last):
File "test.py", line 8, in <module>
do_stuff()
File "test.py", line 4, in do_stuff
print(s[6])
IndexError: list index out of range
$ cat mylogs.txt
Traceback (most recent call last):
File "test.py", line 8, in <module>
do_stuff()
File "test.py", line 4, in do_stuff
print(s[6])
IndexError: list index out of range
This way, you don't need to modify anything with the code.
Note that this solution is for Linux or Mac systems.
See other StackOverflow posts for redirecting Python output to a file.

Unexpected line given when Python interpreter reports exception [duplicate]

This question already has an answer here:
Why does Python read from the current directory when printing a traceback?
(1 answer)
Closed 3 years ago.
When the Python interpreter reports an error/exception (I'm just going to say "error" to refer to both of these from now on), it prints the line number and contents of the line that caused the error.
Interestingly, if you have a long-running Python script which causes an error and change the .py file while the script is running, then the interpreter can report an incorrect line as raising the error, based on the changed contents of the .py file.
MWE:
sample.py
from time import sleep
for i in range(10):
print(i)
sleep(1)
raise Exception("foo", "bar")
This script runs for 10 seconds, then raises an exception.
sample2.py
from time import sleep
for i in range(10):
print(i)
sleep(1)
"""
This
is
just
some
filler
to
demonstrate
the
behavior
"""
raise Exception("foo", "bar")
This file is identical to sample.py except that it has some junk between the end of the loop and the line raises the following exception:
Traceback (most recent call last):
File "sample.py", line 7, in <module>
Exception: ('foo', 'bar')
What I Did
python3 sample.py
In a second terminal window, mv sample.py sample.py.bak && cp sample2.py sample.py before sample.py finishes execution
Expected Behavior
The interpreter reports the following:
Traceback (most recent call last):
File "sample.py", line 7, in <module>
Exception: ('foo', 'bar')
Here, the interpreter reports that there was an exception on line 7 of sample.py and prints the Exception.
Actual Behavior
The interpreter reports the following:
Traceback (most recent call last):
File "sample.py", line 7, in <module>
"""
Exception: ('foo', 'bar')
Here, the interpreter also reports """ when it reports the exception.
It seems to be looking in the file on disk to find this information, rather than the file loaded into memory to run the program.
Source of my Confusion
The following is my mental model for what happens when I run python3 sample.py:
The interpreter loads the contents of sample.py into memory
The interpreter performs lexical analysis, semantic analysis, code generation, etc. to produce machine code
The generated code is sent to the CPU and executed
If an error is raised, the interpreter consults the in-memory representation of the source code to produce an error message
Clearly, there is a flaw in my mental model.
What I want to know:
Why does the Python interpreter consult the file on disk to generate error message, rather than looking in memory?
Is there some other flaw in my understanding of what the interpreter is doing?

As per the answer linked by #b_c,
Python doesn't keep track of what source code corresponds to any compiled bytecode. It might not even read that source code until it needs to print a traceback.
[...]
When Python needs to print a traceback, that's when it tries to find source code corresponding to all the stack frames involved. The file name and line number you see in the stack trace are all Python has to go on
[...]
The default sys.excepthook goes through the native call PyErr_Display, which eventually winds up using _Py_DisplaySourceLine to display individual source lines. _Py_DisplaySourceLine unconditionally tries to find the file in the current working directory (for some reason - misguided optimization?), then calls _Py_FindSourceFile to search sys.path for a file matching that name if the working directory didn't have it.

Multiline script in a python command line argument

I am trying to invoke python from within an arch linux PKGBUILD script:
python -c "from module import func; func()"
The func raises an exception, which is expected behavior but causes the script to fail.
Catching the exception like this does not work:
python -c "from module import func; try: func(); except ValueError: pass"
It seems there is no way to put try/except statements into a single line (Python: try statement in a single line).
Is there another way to ignore the exception or the fact that python returns with an error?
A solution that does not require additional scripts or other files would be most welcome :)

Strings in shell can contain embedded newlines:
python -c 'from module import func
try:
func()
except ValueError:
pass
'
Note that this presents a challenge if the Python to run contains a mix of quotation marks, which would require some contortions to ensure they are all escaped properly. In that case, a here document would be more appropriate then the -c option (although this presents its own issues if the code to run needs to read from standard input.)
python << EOF
from module import func
try:
func()
except ValueError:
pass
EOF

You can also enter the script to be executed interactively like this:
$ cat -- | python
<code here>
<code here>
<code here>
<press Ctrl-D>
and Python will run what you entered, for example:
~$ cat -- | python
from module import func
try:
func()
except ValueError:
pass
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named module