How can I mute particular logs in python using logger? - python

I'm struggling getting something going that should be pretty simple. I'm working on some uniittests that calls code with some very verbose debug logging. How can I tell the logging module "set everything to debug, except this one module, make it info"

disable query logging - Google Appengine SDK Python
check the log filter, which can be used as module based filter.

Related

Python logging, how to prevent functions from library from logging?

I'm making a project and using a library from the requirements of the project. The library implements logging and automatically logs to a file, when I call its functions.
Now, I'm implementing logging by myself, and only want my own messages to be logged to the file and not the library messages.
One solution I thought of would be switching the logging file each time I call a function from the library and then removing that file, but that seems overly complicated and clutterly. What can I do in order to not log the messages from the library?
P.S.:
I'm using the logging library and I initalize it as:
logging.basicConfig(level = logging.INFO,filename = loggingFile,format = "%(message)s")
, which means, all messages from myself and from the library will get logged in loggingFile
Libraries should not directly output anything to logs - that should be done only by handlers configured by an application. A library that logs output is an anti-pattern - if a library definitely does that, I'd log a bug against that library's issue tracker.
On the other hand, it might be that the library is only outputting stuff because you have configured output via your basicConfig() call. If you need more than basic configuration, don't usebasicConfig() - use the other APIs provided.

How to make SQLAlchemy and Flask use the same logger?

probably I don't quite understand how logging really works in Python. I'm trying to debug a Flask+SQLAlchemy (but without flask_sqlalchemy) app which mysteriously hangs on some queries only if run from within Apache, so I need to have proper logging to get meaningful information. The Flask application by default comes with a nice logger+handler, but how do I get SQLAlchemy to use the same logger?
The "Configuring Logging" section in the SQLAlchemy just explains how to turn on logging in general, but not how to "connect" SQLAlchemy's logging output to an already existing logger.
I've been looking at Flask + sqlalchemy advanced logging for a while with a blank, expressionless face. I have no idea if the answer to my question is even in there.
EDIT: Thanks to the answer given I now know that I can have two loggers use the same handler. Now of course my apache error log is littered with hundreds of lines of echoed SQL calls. I'd like to log only error messages to the httpd log and divert all lower-level stuff to a separate logfile. See the code below. However, I still get every debug message into the http log. Why?
if app.config['DEBUG']:
# Make logger accept all log levels
app.logger.setLevel(logging.DEBUG)
for h in app.logger.handlers:
# restrict logging to /var/log/httpd/error_log to errors only
h.setLevel(logging.ERROR)
if app.config['LOGFILE']:
# configure debug logging only if logfile is set
debug_handler = logging.FileHandler(app.config['LOGFILE'])
debug_handler.setLevel(logging.DEBUG)
app.logger.addHandler(debug_handler)
# get logger for SQLAlchemy
sq_log = logging.getLogger('sqlalchemy.engine')
sq_log.setLevel(logging.DEBUG)
# remove any preconfigured handlers there might be
for h in sq_log.handlers:
sq_log.removeHandler(h)
h.close()
# Now, SQLAlchemy should not have any handlers at all. Let's add one
# for the logfile
sq_log.addHandler(debug_handler)
You cannot make SQLAlchemy and Flask use the same logger, but you can make them writing to one place by add a common Handler. And maybe this article is helpful: https://www.electricmonk.nl/log/2017/08/06/understanding-pythons-logging-module/
By the way, if you want to get all logs in one single request, you can set a uniq name for current thread before request, and add the threadName in you logging's formatter.
Answer to my question at EDIT: I still had "echo=True" set on the create_engine, so what I saw was all the additional output on stderr. echo=False stops that but still logs to debug level DEBUG.
Clear all corresponding handlers created by SqlAlchemy:
logging.getLogger("sqlalchemy.engine.Engine").handlers.clear()
The code above should be called after engine created.

How to change logging level in google app engine python production server?

I have tried to search answers for this question online, but in vain. I do see the answer for "How do set the log level in google app engine python dev server", which is useful to know - but if I understand correctly, this doesn't automatically translate to the production environment, right?
Deploying my dev code with hundreds of logging.debug() statements always makes them show up on the production server logs. Is there a simple switch I could flip on the production server to set the logging level and avoid clogging the logs with all debug messages? At least from looking at the Google App Engine's admin console, I haven't found there is a way to do this. This is hard to believe because one would think that App Engine developers would have provisioned a super simple way to do this.
As Paul Collingwood said in his comment, it is easy to set a filter in the Developer Console, in order to reduce visual clutter.
If there are cases in which you do not wish to have the debug logs recorded at all (e.g. while in production), you might like to write a little wrapper function for the logging calls, which checks whether the app is in dev or production, and based on that decides whether to write something to log.
I'm thinking something like:
import logging
class Logger()
def debug(*args, **kwargs):
if not running_in_production(): # needs to be implemented elsewhere.
logging.debug(*args, **kwargs)
def info(*args, **kwargs):
""" any rules here? """
logging.info(*args, **kwargs)
# other functions here.
Then, in your individual files, you could replace import logging with import logger as logging for a drop-in replacement which is adaptable to the environment where it is running - and to any other imaginable factors.

Write to robot framework console from Python

I am a newbie using python and I wanted to ask for your help in showing me how can I print messages from Python into robot framework console.
There are several ways for a python function to send information to the robot logs or to the console. These are all documented in the Robot framework user's guide, in the section titled Logging information.
The cleanest way is to use the logging API, which gives specialized functions for various kinds of logging. For example, to send information to the console you would use logger.console(message).
Using the logging API
Here is a library file that uses this method:
# ExampleKeywords.py
from robot.api import logger
def write_to_console(s):
logger.console(s)
You can use this library in the following manner:
*** Settings ***
| Library | ExampleKeywords.py
*** Test Cases ***
| Use a custom keyword to write to the console
| | Write to console | Hello, world
This will appear in the console only, and will not show up in the logs. If you want the information to show up in the logs you can use logger methods info, warn, debug, or trace. To log an error you would simply throw an exception.
Calling built-in keywords
There are other ways for your custom keywords to send information to the logs. For example, you can get a reference to the BuiltIn library, and directly call the log or log to console keywords like this:
from robot.libraries.BuiltIn import BuiltIn
def write_to_console(s):
BuiltIn().log_to_console("Hello, world")
Using print statements
Finally, you can write information to the logs (but not only to the console) using print statements. You can prefix the string with *<level>* to affect the log level. For example, to print a warning you can do:
print "*WARN* Danger Will Robinson"
Summary
Using the API is arguably the best method to log information from your keywords. However, this is a fairly new API only available since Robot Framework 2.6, so if you are using an older version of Robot you may need to use one of the other techniques.
What it sounds like you're asking for is a way to write a library such that your testcase can print to the console during test execution.
This is pretty easy, you can find it in the RF Manual under Logging Information. The short version is you can log a "WARN" that will appear in both the log and on screen with a print statement, like so:
print "*WARN* This text will show up on the console."
This command can be used to print any message or content on robot console.
from robot.libraries.BuiltIn import BuiltIn
BuiltIn().log_to_console("Message that needs to be printed")

django logging: log is not created

I'm running my app on the GAE development server, with app-engine-patch to run Django.
One of my views is bugged , so I want to log everything that happens.
I added in myapp.views:
import logging
LOG_FILENAME = '/mylog.txt'
logging.basicConfig(filename=LOG_FILENAME,level=logging.DEBUG)
and my function is:
def function(string):
logging.debug('new call')
#do stuff
#logging.debug('log stuff')
My problem is that I can't find the log. When I run my app I get no errors, but the log is not created.
I also tried various paths: /mylog.txt ,mylog.txt , c:\mylog.txt, c:\complete\path\to \my\app\mylog.txt, but it doesn't work.
On the other hand I tried to create and run a separate test:
#test.py
import logging
LOG_FILENAME = '/mylog.txt'
logging.basicConfig(filename=LOG_FILENAME,level=logging.DEBUG)
logging.debug('test')
And the log is created without problems: c:\mylog.txt
I'm not familiar with logging so I don't know if there might be some issues with django, or appengine.
Thanks
You can't write to files on App Engine - thus, any attempt to log to text files is also doomed to failure. Log output will appear on the SDK console in development, or in the logs console in production.
I am guessing the problem is that you put your log configuration in a view. A general rule of thumb for django logging is to set the log configuration in your settings.py.
By default, dev_appserver suppresses the debug log. To turn it on, run it with the --debug option. Note that dev_appserver itself uses the same logger, and will spew lots of debugging stuff you might not care about.
Your logging will print to stderr.

Categories