I wrote a quick webserver using BaseHTTPServer, and it's working nicely, so now I'm trying to implement logging, and I noticed, hey, seems like BaseHTTPServer already has some logging information that it spits out to the stdout, is there a way to implement my logging to also include this stdout.
i.e. Have logging record all information from the stdout.
Note: I am not explicitly printing anything to the console window, when a GET request is made, BaseHTTPServer handles printing this to the console.
Example:
127.0.0.1 - - [02/May/2014 20:51:52] "GET /postTest.html HTTP/1.1" 200 -
127.0.0.1 - - [02/May/2014 20:51:52] "GET /assets/foundation.js HTTP/1.1" 200 -
127.0.0.1 - - [02/May/2014 20:51:52] "GET /assets/bootstrap.css HTTP/1.1" 200 -
127.0.0.1 - - [02/May/2014 20:51:57] "GET /index.html HTTP/1.1" 200 -
127.0.0.1 - - [02/May/2014 20:51:57] "GET /assets/foundation.js HTTP/1.1" 200 -
127.0.0.1 - - [02/May/2014 20:51:57] "GET /assets/bootstrap.css HTTP/1.1" 200 -
According to the BaseHTTPServer documentation, you can override the log_message method to do this. By default, it just writes to stderr, but you can make it write to your logger instead (or have it write to both).
Related
This question already has answers here:
Link to Flask static files with url_for
(2 answers)
Why use Flask's url_for?
(1 answer)
Closed 3 years ago.
I have 2 route functions, one to get all users and one to get a specific user. Both functions render the same template. The first function works fine, the problem is with the second one. When rendering the template it tries to load static files from another directory.
I tried using a different template for each one and the problem remained.
Get all users route:
#app.route('/users')
def list_users():
users = Users.query.all()
return render_template('users.html', users=users)
Get one user route:
#app.route('/users/<username>')
def get_user(username):
user = Users.query.filter_by(name=username).first()
if user:
return render_template('users.html', users=[user])
Template rendered by get_user route function:
127.0.0.1 - - [07/Aug/2019 13:35:56] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [07/Aug/2019 13:35:57] "GET /favicon.ico HTTP/1.1" 404 -
127.0.0.1 - - [07/Aug/2019 13:36:06] "GET /users/Ana HTTP/1.1" 200 -
127.0.0.1 - - [07/Aug/2019 13:36:06] "GET /users/static/vendor/fontawesome-free/css/all.min.css HTTP/1.1" 404 -
127.0.0.1 - - [07/Aug/2019 13:36:06] "GET /users/static/css/sb-admin-2.min.css HTTP/1.1" 404 -
127.0.0.1 - - [07/Aug/2019 13:36:06] "GET /users/static/img/uatronica_black_transparent.png HTTP/1.1" 404 -
127.0.0.1 - - [07/Aug/2019 13:36:06] "GET /users/static/vendor/jquery/jquery.min.js HTTP/1.1" 404 -
127.0.0.1 - - [07/Aug/2019 13:36:06] "GET /users/static/vendor/bootstrap/js/bootstrap.bundle.min.js HTTP/1.1" 404 -
127.0.0.1 - - [07/Aug/2019 13:36:06] "GET /users/static/vendor/jquery-easing/jquery.easing.min.js HTTP/1.1" 404 -
127.0.0.1 - - [07/Aug/2019 13:36:06] "GET /users/static/js/sb-admin-2.min.js HTTP/1.1" 404 -
127.0.0.1 - - [07/Aug/2019 13:36:06] "GET /users/static/vendor/chart.js/Chart.min.js HTTP/1.1" 404 -
127.0.0.1 - - [07/Aug/2019 13:36:06] "GET /users/static/js/demo/chart-area-demo.js HTTP/1.1" 404 -
127.0.0.1 - - [07/Aug/2019 13:36:06] "GET /users/static/js/demo/chart-pie-demo.js HTTP/1.1" 404 -
It is trying to load css files from /users/static/ instead /static/. Why is that?
You are using relative paths in your template, so the final URL is relative to whatever URL you're at. When viewing /users, if the template links to static/css/admin.css, it becomes /users/static/css/admin.css. If the path starts with a /, it is an absolute URL and won't do this.
Instead, use url_for, which generates absolute URLs no matter where you are and how the app is deployed.
<link rel="stylesheet" href="{{ url_for('static', filename='css/admin.css') }}>
This becomes /static/css/admin.css.
Please use the below code.
Get one user route:
#app.route('/users/<username>')
def get_user(username):
user = Users.query.filter_by(name=username).first()
return render_template('users.html', users=user)
Additionally please share the html page users.html for review.
Trying to move my development environment to run on minikube.
The page loads but my page uses websockets on the same port/protocol that the index.html is loaded with (https in this case), and the websockets do no seem to be working correctly.
Here is an example of the correct output when run through nginx / python on my local development box.
127.0.0.1 - - [14/Sep/2018 14:14:35] "GET / HTTP/1.0" 200 -
127.0.0.1 - - [14/Sep/2018 14:14:35] "GET /static/jquery.min.js HTTP/1.0" 200 -
127.0.0.1 - - [14/Sep/2018 14:14:35] "GET /static/socket.io.min.js HTTP/1.0" 200 -
127.0.0.1 - - [14/Sep/2018 14:14:35] "GET /socket.io/?EIO=3&transport=polling&t=MNPIg-N HTTP/1.0" 200 -
127.0.0.1 - - [14/Sep/2018 14:14:35] "GET /favicon.ico HTTP/1.0" 404 -
127.0.0.1 - - [14/Sep/2018 14:14:35] "GET /favicon.ico HTTP/1.0" 404 -
127.0.0.1 - - [14/Sep/2018 14:14:35] "POST /socket.io/?EIO=3&transport=polling&t=MNPIg-o&sid=0570b4fe27f345e9b11858b3acb40a6e HTTP/1.0" 200 -
127.0.0.1 - - [14/Sep/2018 14:14:35] "GET /socket.io/?EIO=3&transport=polling&t=MNPIg-r&sid=0570b4fe27f345e9b11858b3acb40a6e HTTP/1.0" 200 -
127.0.0.1 - - [14/Sep/2018 14:14:35] "POST /socket.io/?EIO=3&transport=polling&t=MNPIg_x&sid=0570b4fe27f345e9b11858b3acb40a6e HTTP/1.0" 200 -
127.0.0.1 - - [14/Sep/2018 14:14:35] "GET /socket.io/?EIO=3&transport=polling&t=MNPIg_w&sid=0570b4fe27f345e9b11858b3acb40a6e HTTP/1.0" 200 -
127.0.0.1 - - [14/Sep/2018 14:14:40] "GET /socket.io/?EIO=3&transport=polling&t=MNPIh0L&sid=0570b4fe27f345e9b11858b3acb40a6e HTTP/1.0" 200 -
127.0.0.1 - - [14/Sep/2018 14:14:45] "GET /socket.io/?EIO=3&transport=polling&t=MNPIiE3&sid=0570b4fe27f345e9b11858b3acb40a6e HTTP/1.0" 200 -
127.0.0.1 - - [14/Sep/2018 14:14:50] "GET /socket.io/?EIO=3&transport=polling&t=MNPIjSI&sid=0570b4fe27f345e9b11858b3acb40a6e HTTP/1.0" 200 -
127.0.0.1 - - [14/Sep/2018 14:14:55] "GET /socket.io/?EIO=3&transport=polling&t=MNPIkgS&sid=0570b4fe27f345e9b11858b3acb40a6e HTTP/1.0" 200 -
Notice how there is a GET every 5 seconds (that's a timer running on the page)
When running on Kubernetes, The page loads and the timer shows up as if the websocket has worked, however I show no logs where the websocket is having a GET or POST after the initial one.
192.168.99.1,172.17.0.7 - - [14/Sep/2018 18:24:03] "GET /static/jquery.min.js HTTP/1.1" 304 1210 0.008244
192.168.99.1,172.17.0.7 - - [14/Sep/2018 18:24:03] "GET /static/socket.io.min.js HTTP/1.1" 304 1210 0.009271
(10) accepted ('172.17.0.7', 34444)
192.168.99.1,172.17.0.7 - - [14/Sep/2018 18:24:04] "GET /socket.io/?EIO=3&transport=polling&t=MNPKrsy HTTP/1.1" 200 379 0.003682
(10) accepted ('172.17.0.7', 34446)
192.168.99.1,172.17.0.7 - - [14/Sep/2018 18:24:04] "GET /favicon.ico HTTP/1.1" 404 1314 0.004694
(10) accepted ('172.17.0.7', 34448)
(10) accepted ('172.17.0.7', 34450)
(10) accepted ('172.17.0.7', 34452)
192.168.99.1,172.17.0.7 - - [14/Sep/2018 18:24:04] "GET /socket.io/?EIO=3&transport=polling&t=MNPKrtD&sid=77d4755c524f47c2948b9c36da007b85 HTTP/1.1" 200 210 0.000749
192.168.99.1,172.17.0.7 - - [14/Sep/2018 18:24:04] "POST /socket.io/?EIO=3&transport=polling&t=MNPKrtB&sid=77d4755c524f47c2948b9c36da007b85 HTTP/1.1" 200 194 0.002632
(10) accepted ('172.17.0.7', 34454)
192.168.99.1,172.17.0.7 - - [14/Sep/2018 18:24:04] "GET /favicon.ico HTTP/1.1" 404 1314 0.002388
The Ingress is setup as follows:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: websitev2-cluster-ip-service
servicePort: 8080
As mentioned before, the websocket does not reside on a different port and it is instanciated in javascript as:
namespace = '/socket';
var socket = io.connect(location.protocol + '//' + document.domain + ':' + location.port + namespace);
Are there any special requirements to get websockets to work? I do not believe i need a special route because the URI needs to be sent to the same location as everything else, and on the same port.
EDIT: MORE DETAILS
My website has a form, that when submitted, executes the following code:
$('form#job').submit(function(event) {
var nameValue = JSON.stringify($(event.target).serializeArray());
console.log(nameValue)
socket.emit('job_submit', {data: nameValue});
return false;
});
On the python side, I have my socket code, which should get hit once a user clicks the submit button.
#socketio.on('job_submit', namespace='/socket')
def job_submit(message):
print('recieved job_submit message from client')
# kick off subjob in celery task
data = unstringify(message)
print('data = {0}'.format(data))
sessiondata = dict(session)
print('sessiondata = {}'.format(sessiondata))
subjobstart.delay(sessiondata, request.sid, data)
In my logs I am not receiving any "recieved job_submit message from client" messages when the submit button is clicked, which means the javascript is trying to send a websocket emit to the python server, but the message is never getting there.
Emits from python to the client page are working as the time is getting updated on the site.
So not exactly sure what changed as it worked on test box, apparently the issue lied in the form validation I was doing. Once I put in a novalidate option to temporarily bypass the form checking, the form was then able to be submitted and I did not have any websocket issues after all.
Hopefully this post can help someone with the code posted instead.
I have recently created a flask app that streams a video to my local network. This is on a raspberry pi. I have attempted to make a log file using this code::
import logging
logging.basicConfig(filename='error.log',level=logging.DEBUG)
When I run the flask app with app.run(), the app will not log anything to the error.log file. But, when I run it with debug=true , then the app logs all info. The error.log file when I run it looks like this:
INFO:werkzeug:10.0.0.127 - - [10/Feb/2018 16:37:40] "GET /video_feed HTTP/1.1" $
INFO:werkzeug:10.0.0.127 - - [10/Feb/2018 16:37:40] "GET /static/css/bootstrap.$
INFO:werkzeug:10.0.0.127 - - [10/Feb/2018 16:39:45] "GET /video_feed HTTP/1.1" $
INFO:werkzeug:10.0.0.127 - - [10/Feb/2018 16:40:08] "GET / HTTP/1.1" 200 -
INFO:werkzeug:10.0.0.127 - - [10/Feb/2018 16:40:08] "GET /video_feed HTTP/1.1" $
INFO:werkzeug:10.0.0.127 - - [10/Feb/2018 16:40:09] "GET /logs HTTP/1.1" 200 -
INFO:werkzeug:10.0.0.127 - - [10/Feb/2018 16:40:10] "GET /invalidpass HTTP/1.1"$
INFO:werkzeug:10.0.0.127 - - [10/Feb/2018 16:40:21] "GET /logs HTTP/1.1" 200 -
INFO:werkzeug:10.0.0.127 - - [10/Feb/2018 16:40:29] "GET / HTTP/1.1" 200 -
INFO:werkzeug:10.0.0.127 - - [10/Feb/2018 16:40:30] "GET /video_feed HTTP/1.1" $
INFO:werkzeug:10.0.0.127 - - [10/Feb/2018 16:40:31] "GET /logout HTTP/1.1" 302 -
INFO:werkzeug:10.0.0.127 - - [10/Feb/2018 16:40:31] "GET /welcome HTTP/1.1" 302$
INFO:werkzeug:10.0.0.127 - - [10/Feb/2018 16:40:31] "GET /login HTTP/1.1" 200 -
I was wondering if it is possible to not have the Debug on when I run my flask app, and still log all information to the error.log file. I have tried switching the level to logging.INFO but that has not worked.
Try setting the logging level to error.
logging.basicConfig(filename='error.log',level=logging.ERROR)
You are setting the log level for the root logger but as mentioned in the flask logging docs flask configures its own logger in the 'flask' logging namespace.
The application logger is available as a property on the application object. Do
app = Flask(__name__)
app.logger.setLevel(logging.DEBUG)
if you'd like to always show logs at DEBUG level.
I couldn't figure out which way I wanted to go with logging that way, so I decided to create my own logging instance, and then log any info I wanted to it, which seemed to work. Although this is a longer and manual process, it let me control what was logged and how it was logged. The code looked similar to this::
#Import requests from flask to log things such as form data and other requests made to the server
from flask import request
#import python logging for logging to files
import logging
from logging.handlers import RotatingFileHandler
#How I logged the information I wanted to log
logInfo = 'Info on Login: ' + 'IP: ' + request.remote_addr
app.logger.info(logInfo)
#Create the logging instance test
handler = RotatingFileHandler('error.log', maxBytes=10000, backupCount=1)
handler.setLevel(logging.INFO)
app.logger.addHandler(handler)
app.logger.info('INFO TEST')
I am new to Django and this question may seem to be easy. In the terminal where "python manage.py runserver" is executed, the following logs periodically appear.
[30/Sep/2015 02:36:02] "GET /messages/check/?_=1443574208652 HTTP/1.1" 200 1
[30/Sep/2015 02:36:08] "GET /notifications/check/?_=1443574208653 HTTP/1.1" 200 1
[30/Sep/2015 02:36:08] "GET /feeds/update/?first_feed=13&last_feed=6&feed_source=all&_=1443574208655 HTTP/1.1" 200 173
[30/Sep/2015 02:36:08] "GET /feeds/check/?last_feed=13&feed_source=all&_=1443574208654 HTTP/1.1" 200 1
[30/Sep/2015 02:36:39] "GET /notifications/check/?_=1443574208656 HTTP/1.1" 200 1
[30/Sep/2015 02:36:39] "GET /feeds/check/?last_feed=13&feed_source=all&_=1443574208658 HTTP/1.1" 200 1
[30/Sep/2015 02:36:39] "GET /feeds/update/?first_feed=13&last_feed=6&feed_source=all&_=1443574208657 HTTP/1.1" 200 173
[30/Sep/2015 02:37:03] "GET /messages/check/?_=1443574208659 HTTP/1.1" 200 1
My Django virtual environment is as follows:
bleach==1.4
dj-database-url==0.3.0
dj-static==0.0.6
Django==1.6.5
gunicorn==19.3.0
html5lib==0.9999999
Markdown==2.4.1
Pillow==2.4.0
psycopg2==2.6.1
python-decouple==2.2
six==1.9.0
South==0.8.4
static3==0.6.1
Unipath==1.0
Questions are:
Why does the logs periodically appear? I guess there is a background task running.But Celery package is not used here.
How to configure the logs, for example, displaying one more logs such as:
/followers/check/?_=9896886900907 HTTP/1.1
What do "?_" and the long string of numbers stand for? I can understand the "?first_feed=13" means the querying parameter in Django request object.
GET /notifications/check/?_=1443574208656
When Django is run in development mode (with runserver) it prints a line for all HTTP requests received to the console.
All of them are coming from the outside, browsers, feed readers...
The parameters you are wondering about are GET query parameters. The ? is the delimiter so, _ is the key of the argument.
In development mode the logging is defined by default. If you want to have logging on your production system or more logging, read https://docs.djangoproject.com/en/1.8/topics/logging/ .
Flask seems to be not adding a slash at the url end before the get parameters in every case. But is doing it only in this case.
it changes /users?uid=1 to /users/?uid=1
After changing it to tha it even gives me a 404 error. "The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again."
Here's the code:
from flask import Flask, render_template, jsonify, Response, request
app = Flask(__name__)
#app.route("/users")
#app.route("/post")
#app.route("/bookmarks")
#app.route("/<target>")
def category_browser(target = ""):
if(target != "" and target not in ['categories']):
return render_template("404.html")
else:
return render_template("view.html")
if(__name__ == "__main__"):
app.debug = True;
app.run(port=int(80))
You had a stale cache entry in instance of Chromium when you had the route defined as #app.route("/users/"). You later changed to #app.route("/users") which Chrome still had it cached with the trailing /. Try accessing this simple example using incognito mode and see that /users?uid=1 remaining unchanged and that no 404 is reported. This is what happens when I first accessed it initially (using Chrome 42).
127.0.0.1 - - [07/Jul/2015 14:02:39] "GET /users?target=1 HTTP/1.1" 200 -
Then stopping that script (thanks for that complete almost self-contained example) and add #app.route("/users/") to the list of routes, below the original #app.route("/users/") route (to have a higher order of precedence so that Flask first trigger the redirect), i.e.:
#app.route("/users")
#app.route("/users/")
(Or simply remove the #app.route("/users") decorator)
Now try accessing the same page again in your incognito session, note that in your console:
127.0.0.1 - - [07/Jul/2015 14:04:11] "GET /users?target=1 HTTP/1.1" 301 -
127.0.0.1 - - [07/Jul/2015 14:04:11] "GET /users/?target=1 HTTP/1.1" 200 -
Ah, there's your redirect. Remove that extra line we just added, try going to /users?target=1 again, this is what happens:
127.0.0.1 - - [07/Jul/2015 14:07:22] "GET /users/?target=1 HTTP/1.1" 404 -
Chrome silently rewrites the URL to /users/?target=1 based on the cache entry in the incognito mode, and is reflected because only that URL is showing up on the Flask access log.
If you wish to support both methods, you have to do it this way:
#app.route("/users/")
#app.route("/users")
Then both access methods work:
127.0.0.1 - - [07/Jul/2015 14:08:49] "GET /users/?target=1 HTTP/1.1" 200 -
127.0.0.1 - - [07/Jul/2015 14:08:59] "GET /users?target=1 HTTP/1.1" 200 -
Rather than resulting in:
127.0.0.1 - - [07/Jul/2015 14:10:00] "GET /users?target=1 HTTP/1.1" 301 -
127.0.0.1 - - [07/Jul/2015 14:10:00] "GET /users/?target=1 HTTP/1.1" 200 -