We Keep Coding

how can i redirect my error message from console to file in Python [duplicate]

This question already has an answer here:
redirect_stderr does not work (Python 3.5)
(1 answer)
Closed last year.
I need to redirect my error message from the console to a file. For this example, I need to insert the error message into a file:
Traceback (most recent call last):
File "C:/Users/", line 5, in <module>
1/0
ZeroDivisionError: division by zero"
I have already tried to do something like this:
from contextlib import redirect_stdout
with open('error.txt', 'w') as f:
with redirect_stdout(f):
1/0
print('here is my error')

If you plan to run your script in console itself, you can just use the bash's ">" operator to send the input of your command (in this situation : your script) in a file just like this :
python ./yourScript > ./outputFile
Everything that your script will print will go in the specified file.

You need to catch the error or your application will fail:
from contextlib import redirect_stdout
with open('error.txt', 'w') as f:
try:
1/0
except ZeroDivisionError as e:
f.write(e)

Note: This assumes you're using Bash. I see that you are using Windows, so it's likely that you aren't using Bash. But from what I've read, this should still be applicable if you are using Cmd.exe, it's just that the syntax might be slightly different.
I think it's better to handle error message output outside of your script. Your script should attempt to do the "happy path" work and print an error to stderr if something goes wrong. This is what should happen by default in every programming language. Python gets this right.
Here is an example script:
print("Dividing by 0 now, I sure hope this works!")
1/0
print("Holy cow, it worked!")
If I run this script, the first line prints to stdout, and then the ZeroDivisionError output prints to stderr:
$ python /tmp/script.py
Dividing by 0 now, I sure hope this works!
Traceback (most recent call last):
File "/tmp/script.py", line 3, in <module>
1/0
ZeroDivisionError: integer division or modulo by zero
If I want to run the script and collect any error output in a file, I can use redirection in my shell when I run the command:
$ python /tmp/script.py 2> /tmp/errors.txt
Dividing by 0 now, I sure hope this works!
$ cat /tmp/errors.txt
Traceback (most recent call last):
File "/tmp/script.py", line 3, in <module>
1/0
ZeroDivisionError: integer division or modulo by zero

How do I print Python errors to a txt file, if I don't know where the error is

I have a very large Python 3.x program running in Windows. It works great 99.9% of the time, but occasionally it crashes. I'm not sure what is causing the crash, it could be numerous things. Due to the fact that I have to run the program "compiled" .exe with an invisible console for security reasons (don't ask), I don't get to see any form of console readout when it dies. So obviously it would be great if I could have it output the crash traceback as a text file instead.
I'm familiar with try/except in Python but the piece of code that's causing the issue could be anywhere and I don't want to have to write an individual try/except around every single line of the literally thousands of lines of code. Is there a way that I can get the program to always output any program-stopping error as a text file, no matter what line of code is causing the problem, or what the error might be?

Somewhere in your code you must have a single entry-point into the code that might be crashing (in the main script, at a minimum). You can wrap that in a try/except pair and then use functions from the traceback module to print the exception to a file when it happens:
Change:
if __name__ == "__main__":
do_stuff()
To:
import traceback
if __name__ == "__main__":
try:
do_stuff()
except:
with open("exceptions.log", "a") as logfile:
traceback.print_exc(file=logfile)
raise
If you want to, you could add some extra code to write extra output to the file, with a time/date stamp or whatever other information you think might be useful. You may want to add additional try/except blocks, more or less like the one above, if you want to give special scrutiny to certain parts of your code. For instance, you could put a block in a loop, where you can print out the loop value if an exception occurs:
for x in some_iterable:
try:
do_something_with(x)
except:
with open("exceptions.log", "a") as logfile:
print("Got an exception while handling {!r} in the loop:".format(x)
traceback.print_exc(file=logfile)
raise # you could omit this line to suppress the exception and keep going in the loop
You could also use the logging module, if you want a more configurable system for the file writing end of the issue. The logging.debug and logging.exception functions both read the same exception information used by the traceback module, but with many more options for formatting things yourself (if you want that). Note that setting up logging is a bit more involved than just opening a file manually.

sometimes you cant use try/except or > in terminal.
you can use sys excepthook.
add this to beginning:
import sys
import traceback
def excepthook(exctype, value, tb):
with open("mylog.txt", "w") as mylog:
traceback.print_exception(exctype, value, tb, file=mylog)
sys.excepthook = excepthook
##########
# your code
after that, all traceback will be print to mylog.txt.

I ended up writing my own logging function
# Takes two inputs - logfile (path to desired .csv), and data to be written
# Writes "Y-M-D|H:M:S, data\n"
f = open(logfile, 'a+')
currentdate = time.strftime('%Y-%m-%d|%H:%M:%S')
f.write(currentdate + ',' + data +'\n')
f.close()
it requires time or datetime, I'm not sure which. Also you need to make sure that the log file exists.
Then I would just plop it wherever I needed, eg: logger(ERRLOG, "OCR didn't find poop. Check {}".format(ocr_outfilepath))

I'm not sure what kind of program this is or how you are running it, but you could try running your Python program and redirecting all its output (or all errors) to a file.
For example, if I have a very-contrived sample Python script like this
def do_stuff():
s = [1, 2, 3, 4, 5]
print(s[6])
if __name__ == "__main__":
do_stuff()
which is deliberately going to raise an IndexError exception.
You could run it like this:
$ python test.py &> mylogs.txt
$ cat mylogs.txt
Traceback (most recent call last):
File "test.py", line 8, in <module>
do_stuff()
File "test.py", line 4, in do_stuff
print(s[6])
IndexError: list index out of range
which redirects all output and errors to a file.
Or, if you can have it displayed on a console and also redirect it to a file:
$ python test.py 2>&1 | tee mylogs.txt
Traceback (most recent call last):
File "test.py", line 8, in <module>
do_stuff()
File "test.py", line 4, in do_stuff
print(s[6])
IndexError: list index out of range
$ cat mylogs.txt
Traceback (most recent call last):
File "test.py", line 8, in <module>
do_stuff()
File "test.py", line 4, in do_stuff
print(s[6])
IndexError: list index out of range
This way, you don't need to modify anything with the code.
Note that this solution is for Linux or Mac systems.
See other StackOverflow posts for redirecting Python output to a file.

Unexpected line given when Python interpreter reports exception [duplicate]

This question already has an answer here:
Why does Python read from the current directory when printing a traceback?
(1 answer)
Closed 3 years ago.
When the Python interpreter reports an error/exception (I'm just going to say "error" to refer to both of these from now on), it prints the line number and contents of the line that caused the error.
Interestingly, if you have a long-running Python script which causes an error and change the .py file while the script is running, then the interpreter can report an incorrect line as raising the error, based on the changed contents of the .py file.
MWE:
sample.py
from time import sleep
for i in range(10):
print(i)
sleep(1)
raise Exception("foo", "bar")
This script runs for 10 seconds, then raises an exception.
sample2.py
from time import sleep
for i in range(10):
print(i)
sleep(1)
"""
This
is
just
some
filler
to
demonstrate
the
behavior
"""
raise Exception("foo", "bar")
This file is identical to sample.py except that it has some junk between the end of the loop and the line raises the following exception:
Traceback (most recent call last):
File "sample.py", line 7, in <module>
Exception: ('foo', 'bar')
What I Did
python3 sample.py
In a second terminal window, mv sample.py sample.py.bak && cp sample2.py sample.py before sample.py finishes execution
Expected Behavior
The interpreter reports the following:
Traceback (most recent call last):
File "sample.py", line 7, in <module>
Exception: ('foo', 'bar')
Here, the interpreter reports that there was an exception on line 7 of sample.py and prints the Exception.
Actual Behavior
The interpreter reports the following:
Traceback (most recent call last):
File "sample.py", line 7, in <module>
"""
Exception: ('foo', 'bar')
Here, the interpreter also reports """ when it reports the exception.
It seems to be looking in the file on disk to find this information, rather than the file loaded into memory to run the program.
Source of my Confusion
The following is my mental model for what happens when I run python3 sample.py:
The interpreter loads the contents of sample.py into memory
The interpreter performs lexical analysis, semantic analysis, code generation, etc. to produce machine code
The generated code is sent to the CPU and executed
If an error is raised, the interpreter consults the in-memory representation of the source code to produce an error message
Clearly, there is a flaw in my mental model.
What I want to know:
Why does the Python interpreter consult the file on disk to generate error message, rather than looking in memory?
Is there some other flaw in my understanding of what the interpreter is doing?

As per the answer linked by #b_c,
Python doesn't keep track of what source code corresponds to any compiled bytecode. It might not even read that source code until it needs to print a traceback.
[...]
When Python needs to print a traceback, that's when it tries to find source code corresponding to all the stack frames involved. The file name and line number you see in the stack trace are all Python has to go on
[...]
The default sys.excepthook goes through the native call PyErr_Display, which eventually winds up using _Py_DisplaySourceLine to display individual source lines. _Py_DisplaySourceLine unconditionally tries to find the file in the current working directory (for some reason - misguided optimization?), then calls _Py_FindSourceFile to search sys.path for a file matching that name if the working directory didn't have it.

How to resolve EOFError: EOF when reading a line?

Code:-
input_var=input("please enter the value")
print(input_var)
Error:-
Enter a value
Runtime Exception
Traceback (most recent call last):
File "file.py", line 3, in
n=input("Enter a value")
EOFError: EOF when reading a line
I have started learning Python and tried to run this simple input and print statement. But its giving me the above error. I have tried running it on a online python compiler and it runs fine but when running on a compiler provided in a learning portal I am getting the above error.

I have tried running it on a online python compiler and it runs fine but when running on a compiler provided in a learning portal I am getting the above error.
input simply reads one line from the "standard input" stream. If the learning portal removes access to it (either closes it or sets it as a non-readable stream) then input is going to immediately get an error when it tries to read from the stream.
It simply means you can't use stdin for anything on that platform, so no input(), no sys.stdin.read(), … (so the resolution is "don't do that", it's pretty specifically forbidden)
In this specific case, the learning platform provides a non-readable stream as stdin e.g. /dev/null:
# test.py
input("test")
> python3 test.py </dev/null
Traceback (most recent call last):
File "test.py", line 4, in <module>
input("test")
EOFError: EOF when reading a line
if stdin were closed, you'd get a slightly different error:
> python3 test.py <&-
Traceback (most recent call last):
File "test.py", line 4, in <module>
input("test")
RuntimeError: input(): lost sys.stdin

How can I redirect only part of a stream?

I'm using Python on bash on Linux. I would like to be able to suppress error messages from a particular module, but keep other error messages. An example would probably be the most efficient way to convey what I want:
File: module.py
import sys
sys.stderr.write('Undesired output.\n')
File: script.py
import module
import sys
sys.stderr.write('Desired output.\n')
sys.stdout.write('Desired output.\n')
x = int('a')
Output of running script.py:
$ python script.py
Undesired output.
Desired output.
Desired output.
Traceback (most recent call last):
File "script.py", line 6, in <module>
x = int('a')
ValueError: invalid literal for int() with base 10: 'a'
Desired output of running script.py:
$ python script.py
Desired output.
Desired output.
Traceback (most recent call last):
File "script.py", line 6, in <module>
x = int('a')
ValueError: invalid literal for int() with base 10: 'a'
I cannot modify module.py, but I must use it. I've tried all kinds of redirects, and opening new file descriptors, but I can't seem to change the file descriptors of the calling shell from within python, so
$ python script.py 2>/dev/null
Desired output.
kills ALL stderr output. Since I know that the module is causing the undesired message, I know exactly which point I want to stop redirecting stderr to /dev/null and start redirecting it to &1, but I can't seem to be able to do it from within Python.
Any advice is much appreciated!

There is an example here in the accepted answer that might help you
Temporarily Redirect stdout/stderr
but in your case, it assumes that you could do
import module
after
import sys
and after you've redirected stderr in your Python code. Like
import sys
... redirect code here
import module
... code to remove redirect
Don't know if that will break your desired functionality, beyond the redirects.
Also you're not supposed to stick import statements in the middle of your code, as it violates the PEP8 style guide.

akubot's answer and some experimentation led me to the answer to this question, so I'm going to accept it.
Do this:
$ exec 3>&1
Then change script.py to be the following:
import sys, os
sys.stderr = open(os.devnull, 'w')
import module
sys.stderr = os.fdopen(3, 'w')
sys.stderr.write('Desired output.\n')
sys.stdout.write('Desired output.\n')
x = int('a')
Then run like this:
$ python script.py 2>/dev/null
yielding the desired output.