This is my code
from telegram.ext import Updater, CommandHandler
import os
from pymongo import MongoClient
TOKEN = 'TOKEN'
def get_db(update, context):
cluster = MongoClient("mongodb+srv://testing:12345678#cluster0.gs9k5.mongodb.net/test?retryWrites=true&w=majority")
result = list(cluster.get_database('DBNAME')['COLLECTIONNAME'].find({}))
update.message.reply_text(str(result))
def main():
updater = Updater(TOKEN, use_context=True)
dp = updater.dispatcher
dp.add_handler(CommandHandler("getdb", get_db))
updater.start_webhook(listen="#.#.#.#",
port=int(PORT),
url_path=TOKEN)
updater.bot.setWebhook('https://MYHEROKUAPP.herokuapp.com/' + TOKEN)
updater.idle()
if __name__ == '__main__':
main()
Everytime I type /getdb, the bot doesn't give me any response. When I tried several experiments, Seems there's some Error on cluster variable. I used try except syntax, but the bot didn't show anything, even from the except and I couldn't found the Error name as well. And I'm using heroku server for the telegram bot. How to fix this?
You can connect to default db (which is the one defined in the connection string) and query the collections like this
client = MongoClient('connect-string')
db = client.get_default_database()
# 'collection_name' is the name of the Mongo collection
list = db.collection_name.find()
I'm not sure if you still have this issue, but the code seems to be okay so far.
Try logging the app info to the terminal to get a better idea of what the error is.
import logging
logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
level=logging.INFO)
logger = logging.getLogger(__name__)
It may be that your MongoDB Atlas is not allowing connection from the server you are running your code from. You can check your cluster to see if it allows access to the DB from your server. You can add the IP address of your server to the cluster to allow it succesfully access the data.
If the collection you're trying to display all items from is large, Telegram will throw an error because the ensuing message will be too long. Ensure that you're running your test with only a few items in your test database.
You should be ale to check your heroku logs or in your terminal to see what other errors might be happening
When sending any message via Flask-Mail, such as below, the last line with mail.send(msg) will also result in the mail headers and content being logged.
Message("Hello", sender="from#example.com", recipients=["to#example.com"])
msg.body = 'anything'
mail.send(msg)
As my mail may contain sensitive information, I want to disable this logging entirely. Yet, playing around with the logging module, I could not find a logger configured for Flask-Mail.
How do I disable logging in Flask-Mail?
Figured this one out:
Flask-Mail uses Python's smtplib to send mail. smtplib does not use the logging module, but it prints information for debugging to stderr.
smtplib includes following method:
def set_debuglevel(self, debuglevel):
"""Set the debug output level.
A non-false value results in debug messages for connection and for all
messages sent to and received from the server.
"""
self.debuglevel = debuglevel
If we use Flask-Mail, we can set this variable when we initialize our app like this:
app = Flask(__name__)
mail = Mail(app)
app.extensions['mail'].debug = 0
Any output is now suppressed.
I would like to log all HTTP requests sent by an aiohttp ClientSession. The docs provide a list of available loggers. So I tried the following:
import asyncio
import logging
import aiohttp
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
http_logger = logging.getLogger("aiohttp.client")
http_logger.setLevel(logging.DEBUG)
http_logger.propagate = True
async def make_request():
async with aiohttp.ClientSession() as session:
async with session.get('https://httpbin.org/get') as resp:
return await resp.text()
loop = asyncio.get_event_loop()
response_text = loop.run_until_complete(make_request())
print(response_text)
But this only produced the following output:
DEBUG:asyncio:Using selector: EpollSelector
// response text print here
I tried all the loggers from that list in the docs and then searched for questions. This one is similar: specify log request format in aiohttp 2
The answer describes how to set up logging for an aiohttp server. Interestingly, they have to explicitly register the logger:
app = web.Application(loop=loop)
app.router.add_get('/', handle)
app.router.add_get('/{name}', handle)
loop.run_until_complete(
loop.create_server(
app.make_handler(access_log=mylogger, #<--------------- HERE
access_log_format='%r %s %b'), '0.0.0.0', 8080))
Is this necessary for the client, too?
The only way where I could reasonably inject my logger would probably be the session. But the API reference for the ClientSession doesn't specify any logger argument.
All that was necessary for me was to import logging and add
logging.basicConfig(level=logging.DEBUG)
in my make_app().
The default access.log format is quite wordy so I calmed it down a bit with the access_log_format argument to web.run_app
web.run_app(app, access_log_format=" :: %r %s %T %t")
Client request-response lifecycle is pretty complex, that's why aiohttp doesn't log it.
Please use client tracing to register callbacks for all steps. After that, you can log all events by any convenient method (print, logging, whatever).
I have a Flask application that works well and produces an occasional error, which is visible when it is running with debug=True:
if __name__ == '__main__':
app.run(debug=True)
I get useful error messages such as:
Traceback (most recent call last):
File "./main.py", line 871, in index_route
KeyError: 'stateIIIII'
I would like to get error messages like these saved to a file when I run the application in production (using Lighttpd + fastcgi).
After looking at various StackOverflow questions (http://flask.pocoo.org/docs/errorhandling/, http://docs.python.org/2/library/logging.html, etc.); the Flask mailing list; and a few blogs, it seems there is no easy way just to send all the great error messages to a file - I need to use the Python logging module to customise things. So I came up with the following code.
At the top of my application file I have various imports followed by:
app = Flask(__name__)
if app.debug is not True:
import logging
from logging.handlers import RotatingFileHandler
file_handler = RotatingFileHandler('python.log', maxBytes=1024 * 1024 * 100, backupCount=20)
file_handler.setLevel(logging.ERROR)
app.logger.setLevel(logging.ERROR)
app.logger.addHandler(file_handler)
I have then put the code for each route in a try/except statement and use traceback to work out which line the error came from and print a nice error message:
def some_route():
try:
# code for route in here (including a return statement)
except:
exc_type, exc_value, exc_traceback = sys.exc_info()
app.logger.error(traceback.print_exception(exc_type, exc_value, exc_traceback, limit=2))
return render_template('error.html')
And then right at the end of the file I remove the debug=True statement. Though I don't think I need to do that as the application is being run by a fastcgi server(?) when it is run in production. The last two lines of my application code look like this:
if __name__ == '__main__':
app.run()
I am struggling to get this working. I think the best I have managed is to get a single error log message to be saved in the file using (app.logger.error('test message') ), but it only prints that one message. An attempt to log another error directly after that one is simply ignored.
I don't know why it's not working but I can tell how am doing this.
First of all, you don't need to set the level of app.logger. So remove this line app.logger.setLevel().
You want to save exception and return error page for every view. It is a lot of work to write this code everywhere. Flask provides a method to do this. Define an errorhandler method like this.
#app.errorhandler(500)
def internal_error(exception):
app.logger.error(exception)
return render_template('500.html'), 500
Whenever a view raises an exception, this method will be called and passed the exception as argument. Python logging provides exception method that is used to save full traceback of the exception.
Since this handles all exception, you don't even need to put code in try/except block. Though, if you want to do something before calling the errorhandler(for e.g. rollback session or transaction) then do this:
try:
#code
except:
#code
raise
If you would like the date and time added for each entry in your log file, the following code can be used (in place of the similar code featured in the question).
if app.debug is not True:
import logging
from logging.handlers import RotatingFileHandler
file_handler = RotatingFileHandler('python.log', maxBytes=1024 * 1024 * 100, backupCount=20)
file_handler.setLevel(logging.ERROR)
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
file_handler.setFormatter(formatter)
app.logger.addHandler(file_handler)
For those who read this later on.
I think it is better idea to push more useful info into error messages. URL, client IP, user-agent etc. Flask logs exceptions internally (in app.debug==False mode) with Flask.log_exception function. So, instead of logging things manually in #app.errorhandler I do something like this:
class MoarFlask(Flask):
def log_exception(self, exc_info):
"""...description omitted..."""
self.logger.error(
"""
Request: {method} {path}
IP: {ip}
User: {user}
Agent: {agent_platform} | {agent_browser} {agent_browser_version}
Raw Agent: {agent}
""".format(
method = request.method,
path = request.path,
ip = request.remote_addr,
agent_platform = request.user_agent.platform,
agent_browser = request.user_agent.browser,
agent_browser_version = request.user_agent.version,
agent = request.user_agent.string,
user=user
), exc_info=exc_info
)
Then, at configuration time, bind FileHandler to app.logger and go on.
I don't use StreamHandler cause many servers (e.g. uWSGI) like to pollute it
with their own proprietary-wordy-useless-not-turnable-off messages.
Don't be afraid of extending Flask. You'll be forced to do it sooner or later ;)
This is what I generally do when a service requires logging. I'm not a specialist on the subject, but these observations are something to take in consideration, in my humble opinion:
at the beginning of every function (route), create a timestamp object, in order to registry the exact time when the request was made, independently if it was successful or not
use #app.after_request, for registering every successful request
use #app.errorhandler, for registering general errors + Tracebacks
Here is an example that demonstrates this idea:
#/usr/bin/python3
""" Demonstration of logging feature for a Flask App. """
from logging.handlers import RotatingFileHandler
from flask import Flask, request, jsonify
from time import strftime
__author__ = "#ivanleoncz"
import logging
import traceback
app = Flask(__name__)
#app.route("/")
#app.route("/index")
def get_index():
""" Function for / and /index routes. """
return "Welcome to Flask! "
#app.route("/data")
def get_data():
""" Function for /data route. """
data = {
"Name":"Ivan Leon",
"Occupation":"Software Developer",
"Technologies":"[Python, Flask, JavaScript, Java, SQL]"
}
return jsonify(data)
#app.route("/error")
def get_nothing():
""" Route for intentional error. """
return foobar # intentional non-existent variable
#app.after_request
def after_request(response):
""" Logging after every request. """
# This avoids the duplication of registry in the log,
# since that 500 is already logged via #app.errorhandler.
if response.status_code != 500:
ts = strftime('[%Y-%b-%d %H:%M]')
logger.error('%s %s %s %s %s %s',
ts,
request.remote_addr,
request.method,
request.scheme,
request.full_path,
response.status)
return response
#app.errorhandler(Exception)
def exceptions(e):
""" Logging after every Exception. """
ts = strftime('[%Y-%b-%d %H:%M]')
tb = traceback.format_exc()
logger.error('%s %s %s %s %s 5xx INTERNAL SERVER ERROR\n%s',
ts,
request.remote_addr,
request.method,
request.scheme,
request.full_path,
tb)
return "Internal Server Error", 500
if __name__ == '__main__':
handler = RotatingFileHandler('app.log', maxBytes=10000, backupCount=3)
logger = logging.getLogger(__name__)
logger.setLevel(logging.ERROR)
logger.addHandler(handler)
app.run(host="127.0.0.1",port=8000)
For more information regarding logrotate and logs on stdout and file at the same time: this Gist
If you are using gunicorn to run your Flask app, you can log all Flask exceptions to the gunicorn logs by adding the gunicorn error handlers to the Flask logger:
In module/__init__.py:
#app.before_first_request
def setup_logging():
if not app.debug:
import logging
gunicorn_logger = logging.getLogger('gunicorn.error')
for handler in gunicorn_logger.handlers:
app.logger.addHandler(handler)
In Development, make sure to set: app.config['PROPAGATE_EXCEPTIONS'] = False. Default is None: https://flask.palletsprojects.com/en/1.1.x/config/
I've just got started using pika(v 0.9.4) with Tornado (through the use of pika.adapters.tornado_connection.TornadoConnection) and I was wondering what's the appropriate way of catching errors when using, say: queue_delete for when the queue you're trying to delete doesn't exist. RabbitMQ raises AMQPError, but I am not sure how this can be handled in an async way.
Anyone has any insights on this ?
Disclaimer: I'm the author of stormed-amqp
I would suggest trying with stormed-amqp
import logging
logging.basicConfig()
from tornado.ioloop import IOLoop
from stormed import Connection
def on_connect():
ch = conn.channel()
ch.queue_declare(queue='hello', durable=False)
ch.queue_declare(queue='hello', durable=True)
def on_error(e):
print "Got Connection error", e.reply_text, e.reply_code
io_loop.stop()
conn = Connection(host='localhost')
conn.on_error = on_error
conn.connect(on_connect)
io_loop = IOLoop.instance()
io_loop.start()
Try to avoid the error. If you declare a connection to the queue, and it doesn't exist, it will be created. Then immediately delete it.
Or, if you will use that queue again in the next week or so, i.e. it is not single-use, then just leave it around and handle deletion as a system admin activity that cleans up long term idle queues.
Or just declare your queues with the auto-delete attribute and they will go away when you disconnect.