I have been stuck on this for the past hour. I had plugged logging into my tkinter gui, but could not get it to work. I then started removing parts until I got at the bare bones example in the very python docs and it will not work. At this point I have nothing else to remove.
The code is as follows:
import logging
LOG_FILENAME = r'logging_example.out'
logging.basicConfig(filename=LOG_FILENAME ,level=logging.DEBUG)
logging.debug('This message should go to the log file')
logging.info('So should this')
logging.warning('And this, too')
f = open(LOG_FILENAME, 'rt')
try:
body = f.read()
finally:
f.close()
print('FILE:')
print (body)
The warning is printed to stdout, but the file is not generated.
I am runing python 3.4, x64 on a windows 7. It is a anacondas distribution, so this is running in Ipython inside spyder.
I guess this should be working
As Jonas Byström noted, this does work outside Ipython. It seems that Ipython configures a logging handler before I get the chance to do so. Also, basicConfig will do nothing if a handler is already present. So, in order to have it working in Ipython, one must do one of three things: 1) Add a new handler, OR 2)reload logging, OR 3) remove existing handlers. I did number 2 bellow.
import logging
from imp import reload
reload(logging)
LOG_FILENAME = r'logging_example.out'
logging.basicConfig(filename=LOG_FILENAME ,level=logging.DEBUG)
logging.debug('This message should go to the log file')
logging.info('So should this')
logging.warning('And this, too')
f = open(LOG_FILENAME, 'rt')
try:
body = f.read()
finally:
f.close()
print('FILE:')
print (body)
See theese for more information:
Logging in ipython;
More on the same
Related
i'm using python 3.8 on ubuntu 20.04, and I cant get the DEBUG statements to log to my other file, my code -
import logging
logging.basicConfig(level=logging.DEBUG, filename = 'logDEBUG.txt')
def say(x):
print(x)
phrase = 'hi'
logging.debug(say(phrase))
You need to return the value, not print it.
import logging
logging.basicConfig(level=logging.DEBUG, filename = 'logDEBUG.txt')
def say(x):
return x
phrase = 'hi'
logging.debug(say(phrase))
Result in logDEBUG.txt:
DEBUG:root:hi
Do you have the problem that the file is not created?
If so check to see what your python path is, i don't know if you expect that the log file should appear in the same folder as your script.
Search for the file on your computer and see where it ends up.
I am trying to save all output (stdout and all errors) of a cell to a file. To save stdout, I am using the following:
import sys
old_stdout = sys.stdout
sys.stdout = open('test.txt', 'w')
print("Hello World! ")
In this case, the output is not displayed and gets saved in the file as expected. To save errors, I used:
#Doesn't work
sys.stderr = open('error.txt','w')
print(a) #Should raise NameError
When I run this cell, I get the error in the notebook, and not in the file as expected:
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-5-de3efd936845> in <module>()
1 #Doesn't work
----> 2 sys.stderr = open('error.txt','w')
3 print("Test")
4 print(a)
NameError: name 'sys' is not defined
I would like this saved in a file and not shown in the notebook. What is the correct code for this?
I think that the problem here is that IPython kernels spawned for the notebook use a ZMQInteractiveShell instance, which catches the errors before they make it to stderr, in order to send the error information to the various potential frontends (consoles, jupyter notebooks, etc). See ipykernel/ipkernel.py#L397-L413 for the code which catches exceptions, then InteactiveShell._showtraceback for the base implementation (print to sys.stderr), and ZMQInteractiveShell._showtraceback for that used by notebook kernels (send stderr-channel messages over zmq to frontends).
If you're not bothered about getting exact stderr output, you could take advantage of IPython's existing error logging, which logs errors to a StreamHandler with the prefix "Exception in execute request:". To use this, set the ipython log level, and alter the provided handler's stream:
import logging
import sys
my_stderr = sys.stderr = open('errors.txt', 'w') # redirect stderr to file
get_ipython().log.handlers[0].stream = my_stderr # log errors to new stderr
get_ipython().log.setLevel(logging.INFO) # errors are logged at info level
Alternatively, to get your shell errors to print directly to a file without alteration, you can monkey-patch the _showtraceback method to print traceback to file as well as zmq message queues:
import sys
import types
# ensure we don't do this patch twice
if not hasattr(get_ipython(), '_showtraceback_orig'):
my_stderr = sys.stderr = open('errors.txt', 'w') # redirect stderr to file
# monkeypatch!
get_ipython()._showtraceback_orig = get_ipython()._showtraceback
def _showtraceback(self, etype, evalue, stb):
my_stderr.write(self.InteractiveTB.stb2text(stb) + '\n')
my_stderr.flush() # make sure we write *now*
self._showtraceback_orig(etype, evalue, stb)
get_ipython()._showtraceback = types.MethodType(_showtraceback, get_ipython())
I am trying to log data to stderr and into a file. The file should contain all log messages, and to stderr should go only the log level configured on the command line. This is described several times in the logging howto - but it does not seem to work for me. I have created a small test script which illustrates my problem:
#!/usr/bin/env python
import logging as l
l.basicConfig(level=100)
logger = l.getLogger("me")
# ... --- === SEE THIS LINE === --- ...
logger.setLevel(l.CRITICAL)
sh = l.StreamHandler()
sh.setLevel(l.ERROR)
sh.setFormatter(l.Formatter('%(levelname)-8s CONSOLE %(message)s'))
logger.addHandler(sh)
fh = l.FileHandler("test.dat", "w")
fh.setLevel(l.DEBUG)
fh.setFormatter(l.Formatter('%(levelname)-8s FILE %(message)s'))
logger.addHandler(fh)
logger.info("hi this is INFO")
logger.error("well this is ERROR")
In line 5th code line I can go for logger.setLevel(l.CRITICAL) or logger.setLevel(l.DEBUG). Both results are unsatisfying.
With logger.setLevel(l.CRITICAL) I get ...
$ python test.py
$ cat test.dat
$
Now with logger.setLevel(l.DEBUG) I get ...
$ python test.py
INFO:me:hi this is INFO
ERROR CONSOLE well this is ERROR
ERROR:me:well this is ERROR
$ cat test.dat
INFO FILE hi this is INFO
ERROR FILE well this is ERROR
$
In one case I see nothing nowhere, in the other I see everything everywhere, and one message is being displayed even twice on the console.
Now I get where the ERROR CONSOLE and ERROR FILE outputs come from, those I expect. I don't get where the INFO:me... or ERROR:me... outputs are coming from, and I would like to get rid of them.
Things I already tried:
Creating a filter as described here: https://stackoverflow.com/a/7447596/902327 (does not work)
Emptying handlers from the logger with logger.handlers = [] (also does not work)
Can somebody help me out here? It seems like a straightforward requirement and I really don't seem to get it.
You can set the root level to DEBUG, set propagate to False and then set the appropriate level for the other handlers.
import logging as l
l.basicConfig()
logger = l.getLogger("me")
# ... --- === SEE THIS LINE === --- ...
logger.setLevel(l.DEBUG)
logger.propagate = False
sh = l.StreamHandler()
sh.setLevel(l.ERROR)
sh.setFormatter(l.Formatter('%(levelname)-8s CONSOLE %(message)s'))
logger.addHandler(sh)
fh = l.FileHandler("test.dat", "w")
fh.setLevel(l.INFO)
fh.setFormatter(l.Formatter('%(levelname)-8s FILE %(message)s'))
logger.addHandler(fh)
logger.info("hi this is INFO")
logger.error("well this is ERROR")
Output:
~$ python test.py
ERROR CONSOLE well this is ERROR
~$ cat test.dat
INFO FILE hi this is INFO
ERROR FILE well this is ERROR
So I exported some unit tests from the Selenium IDE to Python. Now I'm trying to debug something, and I've noticed that Selenium uses the logging module. There is one particular line in selenium.webdriver.remote.remote_connection that I would really like to see the output of. It is:
LOGGER.debug('%s %s %s' % (method, url, data))
At the top of the file is another line that reads:
LOGGER = logging.getLogger(__name__)
So where is this log file? I want to look at it.
In your unit test script, place
import logging
logging.basicConfig(filename = log_filename, level = logging.DEBUG)
where log_filename is a path to wherever you'd like the log file written to.
Without the call to logging.basicConfig or some such call to setup a logging handler, the LOGGER.debug command does nothing.
I am python beginner. My python script logs output to a file (say example.log) using the basic python logging module. However, my python script also makes some 3rd party API calls (for example, parse_the_file) over which I don't have any control. I want to capture the output (usually on console) produced by the API into my example.log. The following example code works partially but the problem is that the contents get overwritten as soon as I start logging output of the API into my log file.
#!/usr/bin/env python
import logging
import sys
import os
from common_lib import * # import additional modules
logging.basicConfig(filename='example.log', filemode='w', level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s')
logging.debug('This is a log message.') # message goes to log file.
sys.stdout = open('example.log','a')
metadata_func=parse_the_file('/opt/metadata.txt') # output goes to log file but OVERWRITES the content
sys.stdout = sys.__stdout__
logging.debug('This is a second log message.') # message goes to log file.
I know there have been post suggesting to similar question on this site but I haven't a workaround/solution for this problem that will work in this scenario.
Try:
log_file = open('example.log', 'a')
logging.basicConfig(stream=log_file, level=logging.DEBUG)
logging.debug("Test")
sys.stdout = log_file
sys.stderr = log_file
stdout_fd = os.dup(1)
stderr_fd = os.dup(2)
os.dup2(log_file.fileno(), 1)
os.dup2(log_file.fileno(), 2)
os.system("echo foo")
os.dup2(stdout_fd, 1)
os.dup2(stderr_fd, 2)
sys.stdout = sys.__stdout__
sys.stderr = sys.__stderr__
However, this will not format it accordingly. If you want that, you can try something like http://plumberjack.blogspot.com/2009/09/how-to-treat-logger-like-output-stream.html