So I am trying to build a restful API using flask, served up by apache on centos (httpd).
Basic API calls work just fine but I am not making much progress on the more advanced aspects because every time it fails I just get an HTTP 500 response which is completely useless for troubleshooting and I have no server-side logs to look at. I am literally trying to solve this through trial and error and it is making me bang my head against the wall.
In order to make any progress on this project I need to setup some basic error logging, but I do not understand the documentation or existing threads about this. It is completely over my head.
What I want to do is have flask write out all warnings and exceptions generated by my application to a specific file (it can be in the app directory to keep it simple).
I am looking for the simplest, easiest, least mind bendy way of doing this... suggestions?
Here is a very simplified version of my app... it shows the basic structure I am using, so please use that for reference.
from flask import Flask, jsonify, request
from flask_restful import reqparse, abort, Resource, Api
app = Flask(__name__)
api = Api(app)
class fetchTicket(Resource):
def post(self):
request_data = request.get_json(force=True)
r_ticket_id = request_data['ticket_id']
return jsonify(ticket_id=r_ticket_id)
api.add_resource(fetchTicket, '/ticket/fetch')
if __name__ == "__main__":
import logging
from logging.handlers import FileHandler
app.debug = True
file_handler = FileHandler("/var/www/project_folder/error.log")
file_handler.setLevel(logging.DEBUG)
app.logger.addHandler(file_handler)
app.run()
But when I run the above code no error.log file is created. I am not sure what I am doing wrong.
Note: I did set the folder permissions so that the apache user has access to write to the directory where the log file should go, as per http://fideloper.com/user-group-permissions-chmod-apache but it did not help so I don't think it is a permissions issue.
You’ll need to explicitly include the stack trace when logging, using the extra kwarg.
logger.exception('Probably something went wrong', extra={'stack': True})
Related
I just came across rollbar and wanted to include it in my Python project.
This is the standard way in which I am told to implement rollbar from the website.
import rollbar
rollbar.init('KEY')
try:
a = s
except:
rollbar.report_exc_info()
Is there a better way to implement this without going through all my try except blocks and replacing them with rollbar.report_exc_info()
Can there be a decorator implementation for this?
My current project is a Flask app that provides API's to end users.
Here's an example for rollbar integration in Flask apps.
https://github.com/rollbar/rollbar-flask-example/blob/master/hello.py
#app.before_first_request
def init_rollbar():
"""init rollbar module"""
rollbar.init(
# access token for the demo app: https://rollbar.com/demo
'fc316ac1f7404dc28af26d5baed1416c',
# environment name
'flasktest',
# server root directory, makes tracebacks prettier
root=os.path.dirname(os.path.realpath(__file__)),
# flask already sets up logging
allow_logging_basic_config=False)
# send exceptions from `app` to rollbar, using flask's signal system.
got_request_exception.connect(rollbar.contrib.flask.report_exception, app)
I have a python server that is currently keeping track of the location of all the buses in my university and generating predictions of arrivals to specific locations.
Now, I wanted to attach a lightweight REST API to this server but I have been running intro problems.
I tried using flask with the following code:
from flask import Flask, jsonify
from PredictionWrapper import *
import threading
class RequestHandler():
def __init__(self,predictionWrapper):
self.app = Flask(__name__)
self.predictor = predictionWrapper
self.app.debug = False
self.app.add_url_rule('/<route>/<int:busStop>','getSinglePrediction',self.getSinglePrediction)
t = threading.Thread(target=self.app.run, kwargs={'host':'0.0.0.0', 'port':80, 'threaded':True})
t.start()
def getSinglePrediction(self, route, busStop):
# TODO Get the actual prediction with given parameters
prediction = self.predictor.getPredictionForStop(route, busStop)
return jsonify({'busStop': busStop, 'prediction': prediction})
def getStopPrediction(self, busStop):
# TODO Get the actual prediction with given parameters
return jsonify({'busStop': busStop, 'prediction': 2})
def run(self):
self.app.run(host='0.0.0.0', port=80, threaded=True)
The problem is that I have been encountering the error below after about half a day of running the server. Note that no requests were made to the server around the time it failed with the following error:
ERROR:werkzeug: - - [01/May/2016 09:55:55] code 400, message Bad request syntax ('\x02\xfd\xb1\xc5!')
After investigating I believe I need to deploy to a WSGI production server. But I have no clue what it means in this specific approach given that 1)the flask server is being threaded in order to run the rest of the prediction application, and 2)I am using classes which none of the documentation uses.
Any help on how to setup the wsgi file with apache, gunicorn, or the technology of your choice would be appreciated. Also, any comments on a better approach on making a non-blocking REST API would be helpful.
Let me know if you need any further clarification!
Not sure if this can actually solve your problem but you can use the coroutine based web server gevent. They have a WSGI server that you can use if that's what you meant by saying that you need to deploy a WSGI production server.
If you want to implement the server to your flask application just do the following:
from gevent.pywsgi import WSGIServer
app = Flask(__name__)
http_server = WSGIServer(('', 5000), app)
http_server.serve_forever()
Gevent in general is a very powerful tool and by issuing context switches as necessary it can handle multiple clients very easily. Also, gevent fully supports flask.
First thing to do would be to put exception handling to deal with bad JSON request data (which maybe is what's happening) something like
def getSinglePrediction(self, route, busStop):
try:
prediction = self.predictor.getPredictionForStop(route, busStop)
return jsonify({'busStop': busStop, 'prediction': prediction})
except:
return jsonify({'busStop': 'error', 'prediction': 'error'})
I'm trying to set up python and flask on the arduino yun. I've managed to run python files via the /etc/config/uhttpd configuration file:
...
list interpreter ".py=/usr/bin/python"
...
The default path for the website's root is: /www in which I've placed a soft link (apps) to the sd card. So now I can run python programs: http://[ip arduino]/apps/helloworld.py
And when I make my first helloflask.py program and run that via python helloflask.py I can see the result at: http://[ip arduino]:5000
But now I want to configure the uhttpd mini webserver (which is capable to exchange information via CGI) to use the flask setup. The URI: http://flask.pocoo.org/docs/deploying/cgi/#server-setup shows some instructions... but I just don't get it. I've made a directory ../apps/uno in which I've placed a __init__.py file with the following content:
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
return "He Flask!"
In the apps dir I've put a file: cgi.py with this content:
from wsgiref.handlers import CGIHandler
from uno import app
CGIHandler().run(app)
Now I when I browse: http://[ip arduino]/cgi.py get a server error occured, contact the administrator (I think this is the CGI interface from uhttpd).
I just don't grasp the CGI configuration for Flask/uhttpd
I looked into this too and got a little further, I was able to setup a simple hello world but once I tried to do something non-trivial I ran into a big issue that uhttpd doesn't support URL rewriting/aliasing. This means your flask app can only be served at the URL of its .py file instead of at a root like http:// (arduino IP) /flaskapp/. None of the routes inside the app will be visible and makes the whole thing unusable.
However, instead of trying to force flask into uhttpd I had great success running the built in server that flask provides. Take a look at this guide I wrote up that uses flask to serve data from a Yun: https://learn.adafruit.com/smart-measuring-cup/overview
The thing to do is add a call to app.run when the script is run, for example make your flask app look like:
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
return "Hello Flask!"
if __name__ == '__main__':
app.run(host='0.0.0.0', debug=True, threaded=True)
Then log in to the Yun and run the script using python. Flask's built in server should start serving the app on http:// (arduino IP) :5000/. Make sure to include the host='0.0.0.0' as it's required to listen on the Yun's external network interface. You probably also want debug=True so there are better error messages (and live reloading of the server when the code changes), and I found threaded=True helps because the default server only handles one connection at a time. The Yun is a relatively slow processor so don't expect to service a lot of concurrent requests, however it's quite capable for providing a simple REST API or web application for a few users.
If you want this server to always run on bootup, edit the /etc/rc.local file to include a call to python and your script.
Is there an elegant way to detect whether my google app engine application has had an error during a request to the application, and write out a much more detailed log when it does?
I have considerable "debug" level logging, but I would like to remove all that and only display the debug information when an error occurs.
If you are using webapp2 I recommend do it during a request with the next code:
app = webapp2.WSGIApplication(debug = os.environ['SERVER_SOFTWARE'].startswith('Dev'), config=config.webapp2_config)
app.error_handlers[404] = handle_error
app.error_handlers[500] = handle_error
routes.add_routes(app)
You can see an example for that handle_error Here :
https://github.com/coto/gae-boilerplate/blob/master/boilerplate/lib/basehandler.py#L70
You need to start from the LogService API docs; Fetch and filter some request_logs (remember to include app_logs); For any req_log you can get the related app_logs. Any app_log have three property (time, level and message). What you looking for is message property.
req_logs = logservice.fetch(start_time=time.time() - 3600,
end_time=time.time(),
minimum_log_level=logservice.LOG_LEVEL_ERROR,
include_app_logs=True)
for req_log in req_logs:
for app_log in req_log.app_logs:
print app_log.message
I've been trying for a few days now to get Google App Engine to run a cron Python script which will simply execute a script hosted on a server of mine.
It doesn't need to post any data to the page, simply open a connection, wait for it to finish then email me.
The code I've previously written has logged as "successful" but I never got an email, nor did I see any of the logging.info code I added to test things.
Ideas?
The original and wrong code that I originally wrote can be found at Google AppEngine Python Cron job urllib - just so you know I have attempted this before.
Mix of weird things was happening here.
Firstly, app.yaml I had to place my /cron handler before the root was set:
handlers:
- url: /cron
script: assets/backup/main.py
- url: /
static_files: assets/index.html
upload: assets/index.html
Otherwise I'd get crazy errors about not being able to find the file. That bit actually makes sense.
The next bit was the Python code. Not sure what was going on here, but in the end I managed to get it working by doing this:
#!/usr/bin/env python
# import logging
from google.appengine.ext import webapp
from google.appengine.api import mail
from google.appengine.ext.webapp.util import run_wsgi_app
from google.appengine.api import urlfetch
import logging
class CronMailer(webapp.RequestHandler):
def get(self):
logging.info("Backups: Started!")
urlStr = "http://example.com/file.php"
rpc = urlfetch.create_rpc()
urlfetch.make_fetch_call(rpc, urlStr)
mail.send_mail(sender="example#example.com",
to="email#example.co.uk",
subject="Backups complete!",
body="Daily backups have been completed!")
logging.info("Backups: Finished!")
application = webapp.WSGIApplication([('/cron', CronMailer)],debug=True)
def main():
run_wsgi_app(application)
if __name__ == '__main__':
main()
Whatever it was causing the problems, it's now fixed.