We Keep Coding

How set path of folder and files in Gunicorn. ( wsgi.py )

this my flask module main.py
import os
import Flask
import pandas as pd
app = Flask(__name__)
#app.route("/") # this route is working
def index():
return "this working"
#app.route("/data", methods=["POST"])
def get_data():
json = request.json
df = pd.DataFrame(json)
#here some other code that work on the data that are geting from folderspath as we have define below
if __name__=="__main__":
path=os.path.join(os.path.abspath(os.path.join('data')))
folder=os.path.join(path,'test')
app.run(debug=Flase,host="0.0.0.0")
if we just run the flask server it work and execute path statment but if we set for deployment purpose and using gunicorn the first route work. but when we send request to second route it give error of missing folder that are mention in paths. the below module (wsgi.py) is not getting those path how to set these path, that work in wsgi.py
my Gunicorn file wsgi.py
from main import app
if __name__=="__main__":
app.run()
i want wsgi.py to execute those path before app.run() i tried to put in those statement in wsgi before app.run() and imported dependences but still not working.
any help would be appreciated. thanks

You could try gunicorn's --pythonpath argument:
--pythonpath STRING A comma-separated list of directories to add to the Python path.

How to deploy a Flask application on pythonanywhere with a folder as a module

Currently I am trying to a deploy my first FLASK application on PythonAnywhere.
Im not sure if this is the correct terminology but I have a folder as a module and there for I can't seem to find the correct way to deploy my application. I am not even sure where to start in resolving this issue. Any advice?
File and Folder Layout Snipped
my init.py code is:
import os
from flask import Flask
def create_app(test_config=None):
# create and configure the app
app = Flask(__name__, instance_relative_config=True)
app.config.from_mapping(
SECRET_KEY='secret',
DATABASE=os.path.join(app.instance_path, 'LAMA.sqlite'),
)
if test_config is None:
# load the instance config, if it exists, when not testing
app.config.from_pyfile('config.py', silent=True)
else:
# load the test config if passed in
app.config.from_mapping(test_config)
# ensure the instance folder exists
try:
os.makedirs(app.instance_path)
except OSError:
pass
# database
from . import db
db.init_app(app)
# authentication blueprint
from . import auth
app.register_blueprint(auth.bp)
# blog blueprint - the main index
# from . import blog
# app.register_blueprint(blog.bp)
# app.add_url_rule('/', endpoint='index')
# book blueprint
from . import book
app.register_blueprint(book.bp)
app.add_url_rule('/', endpoint='index')
return app
I have also followed the python debugging page where I have done the following:
>>> import LAMA
>>> print(LAMA)
<module 'LAMA' from '/home/ivanv257/LAMA_MAIN/LAMA/__init__.py'>
So at this stage in my WSGI configuration file I have:
import sys
path = '/home/ivanv257/LAMA_MAIN/LAMA/__init__.py'
if path not in sys.path:
sys.path.append(path)
from LAMA import app as application
I have also tried many other combinations such as
path = '/home/ivanv257/LAMA_MAIN/LAMA/'
from init import app as application
path = '/home/ivanv257/LAMA_MAIN/'
from init import app as application
path = '/home/ivanv257/LAMA_MAIN/'
from LAMA import app as application
my source code path is : /home/ivanv257/LAMA_MAIN/LAMA , although I have also tried different combinations such as /home/ivanv257/LAMA_MAIN/
ERROR DETAIL:
2018-12-08 10:05:32,028: Error running WSGI application
2018-12-08 10:05:32,030: ModuleNotFoundError: No module named 'LAMA'
2018-12-08 10:05:32,030: File "/var/www/ivanv257_pythonanywhere_com_wsgi.py", line 83, in <module>
2018-12-08 10:05:32,030: from LAMA import app as application # noqa

To solve my problem I changed the following (with some assistance) from lama import create_app:
import sys
path = '/home/ivanv257/LAMA_MAIN/LAMA'
if path not in sys.path:
sys.path.append(path)
from lama import create_app
application = create_app()
I also had to remove the from . to just imports
import db
db.init_app(app)
# authentication blueprint
import auth
app.register_blueprint(auth.bp)

You are close. To deploy your app, navigate to the webapps page on your user dashboard. If you have not done so already, click the "Add new a webapp" button and enter the desired name of the app. Then, on the same webapp page on the dashboard, scroll down to the "Code" section of the page. Click on the "source code" a href and add the absolute path (full path) to the lama_main directory storing your init.py file.
Next, click on "WSGI configuration file" link. In the WSGI configuration file for your app, set the correct path to the parent directory and import app from the init.py file:
import sys
# add your project directory to the sys.path
project_home = u'/home/your_user_name/lama_main'
if project_home not in sys.path:
sys.path = [project_home] + sys.path
# import flask app but need to call it "application" for WSGI to work
from init import app as application #note that the module being imported from must be the file with "app" defined.
Then, save the WSGI file and return to the webapp panel on your dashboard. Click the "Reload {your site name}" button. Now, you should be able to visit the site by clicking on the main link at the top of the page.

How can I make a Python package which serves a website with Flask to work with Apache?

I have written a Python package hwrt (see installation instructions if you want to try it) which serves a website when executed with
$ hwrt serve
2014-12-04 20:27:07,182 INFO * Running on http://127.0.0.1:5000/
2014-12-04 20:27:07,183 INFO * Restarting with reloader
I would like to let it run on http://www.pythonanywhere.com, but when I start it there I get
19:19 ~ $ hwrt serve
2014-12-04 19:19:59,282 INFO * Running on http://127.0.0.1:5000/
Traceback (most recent call last):
File "/home/MartinThoma/.local/bin/hwrt", line 108, in <module>
main(args)
File "/home/MartinThoma/.local/bin/hwrt", line 102, in main
serve.main()
File "/home/MartinThoma/.local/lib/python2.7/site-packages/hwrt/serve.py", line 95, in main
app.run()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 739, in run
run_simple(host, port, self, **options)
File "/usr/local/lib/python2.7/dist-packages/werkzeug/serving.py", line 613, in run_simple
test_socket.bind((hostname, port))
File "/usr/lib/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 98] Address already in use
I only found this in the documentation:
Flask
never use app.run(), it will break your webapp. Just import the
app into your wsgi file...
By searching for wsgi file, I found mod_wsgi (Apache). However, I don't understand how I can adjust my current minimalistic Flask application to work with that. Currently, the script behind hwrt serve is:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""Start a webserver which can record the data and work as a classifier."""
import pkg_resources
from flask import Flask, request, render_template
from flask_bootstrap import Bootstrap
import os
import json
# hwrt modules
import hwrt
import hwrt.utils as utils
def show_results(results, n=10):
"""Show the TOP n results of a classification."""
import nntoolkit
classification = nntoolkit.evaluate.show_results(results, n)
return "<pre>" + classification.replace("\n", "<br/>") + "</pre>"
# configuration
DEBUG = True
template_path = pkg_resources.resource_filename('hwrt', 'templates/')
# create our little application :)
app = Flask(__name__, template_folder=template_path)
Bootstrap(app)
app.config.from_object(__name__)
#app.route('/', methods=['POST', 'GET'])
def show_entries():
heartbeat = request.args.get('heartbeat', '')
return heartbeat
#app.route('/interactive', methods=['POST', 'GET'])
def interactive():
if request.method == 'POST':
raw_data_json = request.form['drawnJSON']
# TODO: Check recording
# TODO: Submit recorded json to database
# Classify
model_path = pkg_resources.resource_filename('hwrt', 'misc/')
model = os.path.join(model_path, "model.tar")
print(model)
results = utils.evaluate_model_single_recording(model, raw_data_json)
# Show classification page
page = show_results(results, n=10)
page += 'back'
return page
else:
# Page where the user can enter a recording
return render_template('canvas.html')
def get_json_result(results, n=10):
s = []
for res in results[:min(len(results), n)]:
s.append({res['semantics']: res['probability']})
return json.dumps(s)
#app.route('/worker', methods=['POST', 'GET'])
def worker():
# Test with
# wget --post-data 'classify=%5B%5B%7B%22x%22%3A334%2C%22y%22%3A407%2C%22time%22%3A1417704378719%7D%5D%5D' http://127.0.0.1:5000/worker
if request.method == 'POST':
raw_data_json = request.form['classify']
# TODO: Check recording
# TODO: Submit recorded json to database
# Classify
model_path = pkg_resources.resource_filename('hwrt', 'misc/')
model = os.path.join(model_path, "model.tar")
results = utils.evaluate_model_single_recording(model, raw_data_json)
return get_json_result(results, n=10)
else:
# Page where the user can enter a recording
return "Classification Worker (Version %s)" % hwrt.__version__
def get_parser():
"""Return the parser object for this script."""
from argparse import ArgumentParser, ArgumentDefaultsHelpFormatter
parser = ArgumentParser(description=__doc__,
formatter_class=ArgumentDefaultsHelpFormatter)
return parser
def main():
app.run()
if __name__ == '__main__':
main()

Ok, a not so non-sequitur answer to your question is around what mod_wsgi does to interface with your app. A typical flask app would look something like this:
from flask import Flask
app = Flask(__name__)
app.route("/")
def hello():
return "Holy moly that tunnel was bright.. said Bit to NIC"
if __name__ == "__main__":
app.run()
Unfortunately, Apache has no way to know what to do with this (though the app would run happily on its own). In order to get the app and Apache to play nice together we're going to use something called mod_wsgi. What Mod_WSGI does that's important to us, is that it provides a known interface (a file type called wsgi) that's going to wrap our application and initialize it so that we can serve it through Apache.
I'm going to assume you are using a python virtual environment, but if you aren't you can omit the step that deals with this in the instructions below. If you're curious why virtual environments are so great, feel free read about the python ecosystem.
Also - you can include an extra flag (assuming you are running wsgi as a daemon) to automatically reload the daemon whenever you touch or alter your wsgi file. This is quite useful during development and debugging so I'll include is below.
Anyway, let's get started. I'll break this down to steps below.
Configuring Apache for mod_wsgi
Enable mod_wsgi in Apache:
sudo apt-get install libapache2-mod-wsgi
Edit your /etc/apache2/sites-available/<yoursite>.conf.
<VirtualHost interface:port>
WSGIDaemonProcess yourapp user=someUser processes=2 threads=15
WSGIProcessGroup yourapp
# In this case / refers to whatever relative URL path hosts flask
WSGIScriptAlias / /absolute/path/to/yourapp.wsgi
<Directory /path/to/your/main/py/file/ >
# Use good judgement here when server hardening, this assumes dev env
Order allow,deny
Allow from all
Require all granted
#The below enables 'auto-reload' of WSGI
WSGIScriptReloading On
</Directory>
# If you want to serve static files as well and bypass flask in those cases
Alias /relative/url/to/static/content/
<Directory /absolute/path/to/static/root/directory/>
Order allow,deny
Allow from all
</Directory>
</VirtualHost>
Create your yourapp.wsgi file and put it in the appropriate place: Be wary of file permissions!
#!/usr/bin/python
import sys
import logging
# Activate virtual environment.
# If you are not using venv, skip this.
# But you really should be using it!
activate_this = "/path/to/venv/bin/activate_this.py"
execfile(activate_this, dict(__file__=activate_this))
# Handle logging
logging.basicConfig(stream=sys.stderr)
sys.path.insert(0, "/path/to/your/main/py/file/")
from YourMainPyFileName import app as application
application.secret_key = "your_secret_key"
Reload Apache and troubleshoot problems. I set this up probably every few weeks for a different project or idea I have and... I usually have to fix one thing or another when doing it from scratch. Don't despair though! Flask has great documentation on this.
Once you've done all this you should be at a place where flask runs all on its own. The sample flask app above is the actual code I use to verify everything works whenever I set this up.
This was left here in case it's some use, but is not really directly related to the question...
The answer here is to use x-send-file. This takes advantage of letting Apache do what it's good at (serving static content), while at the same time first letting flask (or other python framework) do it's work first. I do this often to let flask handle my auth layers in single page web apps and have so far been happy with the results.
Doing so requires two things:
First - Enable xsendfile on Apache2 sudo apt-get install libapache2-mod-xsendfile.
Second - Alter your apache2 configuration so allow x-send-file headers:
Alter your conf file in /etc/apache2/sites-available/<yoursite>.conf and add...
XSendFile On
XSendFilePath /path/to/static/directory
This can be entered top level within the <Virtualhost></Virtualhost> tags.
Don't forget to restart Apache sudo service apache2 restart.
Finally - Configure your flask app to use x-send-file in your app.py file:
app.user_x_sendfile = True
Note: Must be done after app initialization. Consequently can also be passed as an initialization parameter.
Flask has documentation on this (excerpt below):
use_x_sendfile
Enable this if you want to use the X-Sendfile feature. Keep in mind that the server has to support this. This only affects files sent with the send_file() method.
New in version 0.2.
This attribute can also be configured from the config with the USE_X_SENDFILE configuration key. Defaults to False.

I ran into a similar issue #moose was having. Getting connection refused and couldnt even telnet localhost 5000.
Turns out theres a ports.conf file i had to add Listen 5000
Happy days.

How to serve static files from my CherryPy applications

I have written a simple web app and I got stuck trying to figure out how to server static files. My static files folder is in a different folder /usr/lib/python2.6/site-packages/web3/static Below is my code, how do I add this static files folder configuration to my app.
#!/usr/bin/env python
import sys
import my_web.settings
from django.core.management import execute_from_command_line
from django.core.management import call_command
import os
import tempfile
import cherrypy.wsgiserver
import django.core.handlers.wsgi
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "my_web.settings")
# Since we use a throwaway database, it needs to be initialized
# every time Django starts.
if sys.argv[1] in ('runserver', 'shell'):
with tempfile.NamedTemporaryFile() as dbFile:
my_web.settings.DATABASES['default']['NAME'] = dbFile.name
call_command('syncdb', interactive=False)
server = cherrypy.wsgiserver.CherryPyWSGIServer(
('0.0.0.0', 8080), django.core.handlers.wsgi.WSGIHandler(),
server_name='localhost', numthreads = 10
)
try:
server.start()
except KeyboardInterrupt:
server.stop()

try this... and be sure the user running the app has permission to access the static path.
cherrypy.config.update({'tools.staticdir.on': True,
'tools.staticdir.dir': '/usr/lib/python2.6/site-packages/web3/static'
})
Hope this helps!

Sylvain Hellegouarch has posted a Django CherryPy integration recipe that nicely solves the problem of serving static Django content. See https://bitbucket.org/Lawouach/cherrypy-recipes/src/c8290261eefb82cb5694930f7236606082a941ff/frameworks/django_/?at=default
Just drop the three python files (init.py, httplogger.py, and djangoplugin.py) into the directory above your app and change the name of the directory holding your settings.py file in init.py to match the directory in your own source code tree.
I also removed the two WebSocketPlugin lines from init.py since that was not relevant for my django app.
There is some discussion about the recipe on Sylvain's blog at http://www.defuze.org/archives/262-hosting-a-django-application-on-a-cherrypy-server.html

Is there an alternative of RewriteRule / .htaccess for a Python http.server.HTTPServer?

Can my server module (with http.server.HTTPServer) use something like the RewriteRule for redirect all traffic into a single cgi script? I'd like to be able to do what's shown here in this other question, but for my python server.
Can it be done using something like .htaccess, or is there another way?
Also, can this be done even for a simple localhost development server?
I am serving files for development via, for example, http://localhost:8000/html/index.html, and I would like to hide the /html subfolder from the URL even in development.
How can that be achieved?

You can use a custom script to initialize your server and define your Routes in it, such as suggested in this article:
Python 2:
server.py
import os
import posixpath
import urllib
import BaseHTTPServer
from SimpleHTTPServer import SimpleHTTPRequestHandler
# modify this to add additional routes
ROUTES = ( <- this is the "good stuff", make aliases for your paths
# [url_prefix , directory_path]
['/media', '/var/www/media'],
['', '/var/www/site'] # empty string for the 'default' match
)
class RequestHandler(SimpleHTTPRequestHandler):
def translate_path(self, path):
"""translate path given routes"""
# set default root to cwd
root = os.getcwd()
# look up routes and set root directory accordingly
for pattern, rootdir in ROUTES:
if path.startswith(pattern):
# found match!
path = path[len(pattern):] # consume path up to pattern len
root = rootdir
break
# normalize path and prepend root directory
path = path.split('?',1)[0]
path = path.split('#',1)[0]
path = posixpath.normpath(urllib.unquote(path))
words = path.split('/')
words = filter(None, words)
path = root
for word in words:
drive, word = os.path.splitdrive(word)
head, word = os.path.split(word)
if word in (os.curdir, os.pardir):
continue
path = os.path.join(path, word)
return path
if __name__ == '__main__':
BaseHTTPServer.test(RequestHandler, BaseHTTPServer.HTTPServer)
Then run your script:
python server.py
Python 3:
In Python 3 the BaseHTTPServer and the SimpleHTTPServer modules have been merged into the http.server module. So you will have to modify the above script as follows:
Change the
import BaseHTTPServer
from SimpleHTTPServer import SimpleHTTPRequestHandler
to
import http.server
(if you do this then the calls should be modified to http.server.SimpleHTTPRequest etc.)
or
from http.server import BaseHTTPServer, SimpleHTTPServer, SimpleHTTPRequestHandler
(with this option calls remain the same as the original script)
then run the server script: python server.py
IN YOUR CASE:
You should modify the ROUTES variable to suite your needs. For example, you want to hide the /html folder from your url:
ROUTES = (
['', '/exact/path/to/folder/html'],
['/another_url_path', '/exact/path/to/another/folder'],
...
)
Now if you hit: http://localhost:8000/index.html you will be in your home page.
Note:
This script by default will serve the contained files in the folder that is in when executed to the domain url (ex. I have server.py on the Documents folder, then when I run it, http://localhost:8000 url will server my Documents folder).
You can change this behavior by the Routes (see the 'default' match comment on the code) or by placing the script in your projects root folder and start it from there.

John Moutafis's answer was helpful to get me started, but needed some fine tuning to run on python3, beyond his comments about imports.
The imports should be
from http.server import HTTPServer, SimpleHTTPRequestHandler
and you'll also need to change the urllib import to:
from urllib.parse import unquote
Then the main should be something like:
if __name__ == '__main__':
myServer = HTTPServer(('0.0.0.0', 8000), RequestHandler)
print("Ready to begin serving files.")
try:
myServer.serve_forever()
except KeyboardInterrupt:
pass
myServer.server_close()
print("Exiting.")