We Keep Coding

How to create a specific logger for my message?

I have spent far too much time on this and I couldn't figure out by myself.
I am writing some code that use some modules, so I want to create a specific logger for my message, not those from the other modules (they are far too many so I don't want to set a different level for each of them).
I have simplified my use case to:
import logging
logger = logging.getLogger('test')
logger.setLevel(level=logging.INFO)
print(logging.Logger.manager.loggerDict)
logger.debug('This is a debug message')
logger.info('This is an info message')
logger.warning('This is a warning message')
logger.error('This is an error message')
logger.critical('This is a critical message')
When I execute the previous code, I would expect to have my info message logged.
Instead I got:
python3 test.py
{'test': <Logger test (INFO)>}
This is a warning message
This is an error message
This is a critical message
Same with DEBUG, it always print warning/error/critical messages, not the debug/info.
If I raise the logging level (let's say ERROR), it works as expected.
python --version
Python 3.9.1

Change the configuration at the package level / root logger (documentation).
logging.basicConfig(level=logging.INFO)

Using the logging module to record the events:
import logging
#Creating an object
logger=logging.getLogger()
#Setting the threshold of logger to DEBUG
logger.setLevel(logging.DEBUG)
#Test messages
logger.debug("Debug Message")
logger.info("An information")
logger.warning("Warning")
logger.error("Error message")
logger.critical("critical Message")

AWS Glue Python ETL: logger messages appear in the error cloudwatch stream

I'm writing a Glue ETL, and I'm trying to log using Python default's logger.
The problem is that all the log messages I'm printing using the logger appear in the error stream of the job.
If I print directly to stdout (using print), I see the printed messages in the regular cloudwatch log stream.
I tried to redirect my logger to stdout, but I still got the same result: messages appear in error stream.
Does anyone know how I can use logger, and still see my messages in the log cloudwatch stream? (and not in the error cloudwatch stream)
This is the code sample I'm using to test:
import logging
import sys
MSG_FORMAT = '%(asctime)s %(levelname)s %(name)s: %(message)s'
DATETIME_FORMAT = '%Y-%m-%d %H:%M:%S'
logging.basicConfig(format=MSG_FORMAT, datefmt=DATETIME_FORMAT, stream=sys.stdout)
logger = logging.getLogger()
logger.setLevel(logging.INFO)
logger.info("Test log message. This appear on the job error cloudwatch stream")
print("This is a print. This appear on the job log cloudwatch stream")

I ended up adding
logging.getLogger().addHandler(logging.StreamHandler(sys.stdout))
to my logger definition.
That resolved the problem

Python - Logging from multiple modules to rotating files without printing to console

I am trying to add logging to a medium size Python project with minimal disruption. I would like to log from multiple modules to rotating files silently (without printing messages to the terminal). I have tried to modify this example and I almost have the functionality I need except for one issue.
Here is how I am attempting to configure things in my main script:
import logging
import logging.handlers
import my_module
LOG_FILE = 'logs\\logging_example_new.out'
#logging.basicConfig(level=logging.DEBUG)
# Define a Handler which writes messages to rotating log files.
handler = logging.handlers.RotatingFileHandler(LOG_FILE, maxBytes=100000, backupCount=1)
handler.setLevel(logging.DEBUG) # Set logging level.
# Create message formatter.
formatter = logging.Formatter('%(asctime)s %(name)-12s %(levelname)-8s %(message)s')
# Tell the handler to use this format
handler.setFormatter(formatter)
# Add the handler to the root logger
logging.getLogger('').addHandler(handler)
# Now, we can log to the root logger, or any other logger. First the root...
logging.debug('Root debug message.')
logging.info('Root info message.')
logging.warning('Root warning message.')
logging.error('Root error message.')
logging.critical('Root critical message.')
# Use a Logger in another module.
my_module.do_something() # Call function which logs a message.
Here is a sample of what I am trying to do in modules:
import logging
def do_something():
logger = logging.getLogger(__name__)
logger.debug('Module debug message.')
logger.info('Module info message.')
logger.warning('Module warning message.')
logger.error('Module error message.')
logger.critical('Module critical message.')
Now, here is my problem. I currently get the messages logged into rotating files silently. But I only get warning, error, and critical messages. Despite setting handler.setLevel(logging.DEBUG).
If I uncomment logging.basicConfig(level=logging.DEBUG), then I get all the messages in the log files but I also get the messages printed to the terminal.
How do I get all messages above the specified threshold to my log files without outputing them to the terminal?
Thanks!
Update:
Based on this answer, it appears that calling logging.basicConfig(level=logging.DEBUG) automatically adds a StreamHandler to the Root logger and you can remove it. When I did remove it leaving only my RotatingFileHandler, messages no longer printed to the terminal. I am still wondering why I have to use logging.basicConfig(level=logging.DEBUG) to set the message level threshold, when I am setting handler.setLevel(logging.DEBUG). If anyone can shed a little more light on these issues it would still be appreciated.

You need to call set the logging level on the logger itself as well. I believe by default, the logging level on the logger is logging.WARNING
Ex.
root_logger = logging.getLogger('')
root_logger.setLevel(logging.DEBUG)
# Add the handler to the root logger
root_logger.addHandler(handler)
The loggers log level determines what the logger will actually log (i.e what messages will actually get handed to the handlers). The handlers log level determines what it will actually handle (i.e. what messages actually are output to file, stream, etc). So you could potentially have multiple handlers attached to a logger each handling a different log level.
Here's an SO answer that explains the way this works

Python logging sometimes writing message to file but not format

Having a new problem where the logger is writing about every other line without the format, just the message.
My code:
import logging
from logging.handlers import RotatingFileHandler
# Set up logging
LOG_FILE = argv[0][:-3] + '.log'
logging.basicConfig(
filename=LOG_FILE,
filemode='a',
level=logging.INFO,
format='%(asctime)s - %(levelname)s - %(message)s')
logger = logging.getLogger(__name__)
handler = RotatingFileHandler(LOG_FILE,maxBytes=1000000)
logger.addHandler(handler)
def main():
target, command, notify_address, wait = get_args(argv)
logger.info('Checking status of %s every %d minutes.' % (target, wait))
logger.info('Running %s and sending output to %s when online.' % (command, notify_address))
The vars returned from get_args() are all strings, even though wait is a number.
Note that I am not receiving any errors in my IDE or when running.
The output I am getting in my log file:
2015-02-12 16:26:27,483 - INFO - Checking status of <ip address> every 30 minutes.
Running <arbitrary bash command string> and sending output to <my email address> when online.
2015-02-12 16:26:27,483 - INFO - Running <arbitrary bash command string> and sending output to <my email address> when online.
What is causing the second logger.info() to print twice, and only once formatted properly?
I have another script that logs perfectly, no idea what I've done here. (Copy/pasted the logging setup section to be safe)

Are you using loggers at different levels of your code? It sounds like the log messages could be propagating upwards. Try adding
logger.propagate = False
after you add the handler. You can check out the python docs for a more detailed explanation here, but the relevant text below sounds exactly like what you're seeing.
Note If you attach a handler to a logger and one or more of its ancestors, it may emit the same record multiple times. In general, you should not need to attach a handler to more than one logger - if you just attach it to the appropriate logger which is highest in the logger hierarchy, then it will see all events logged by all descendant loggers, provided that their propagate setting is left set to True. A common scenario is to attach handlers only to the root logger, and to let propagation take care of the rest.

Python: logging.streamhandler is not sending logs to stdout

I want to use StreamHandler logging handler of python.
What i have tried is,
import logging
import sys
mylogger = logging.getLogger("mylogger")
h1 = logging.StreamHandler(stream=sys.stdout)
h1.setLevel(logging.DEBUG)
mylogger.addHandler(h1)
# now trying to log with the created logger
mylogger.debug("abcd") # <no output>
mylogger.info("abcd") # <no output>
mylogger.warn("abcd") # abcd
Am i missing something ? Or doing any wrong ?
Why INFO and DEBUG level logs are not coming on STDOUT ?

You have to set the level of the logger, not only the level of the handler:
mylogger.setLevel(logging.DEBUG)
Here is a nice graphic of the logging workflow, where you can see that either the logger and the handler check for the log level:
http://docs.python.org/2/howto/logging.html#logging-flow
The default logLevel is WARNING, so even if you set the level of your handler to DEBUG, the message will not get through, since your logger suppresses it (it's also by default WARNING).
By the way, you can do some basic formatting with Formatter:
import logging
import sys
mylogger = logging.getLogger("mylogger")
formatter = logging.Formatter('[%(levelname)s] %(message)s')
handler = logging.StreamHandler(stream=sys.stdout)
handler.setFormatter(formatter)
handler.setLevel(logging.DEBUG)
mylogger.addHandler(handler)
mylogger.setLevel(logging.DEBUG)
mylogger.debug("This is a debug message.")
mylogger.info("Some info message.")
mylogger.warning("A warning.")
Will give you the output
[DEBUG] This is a debug message.
[INFO] Some info message.
[WARNING] A warning.