We Keep Coding

How to deploy a Flask application on pythonanywhere with a folder as a module

Currently I am trying to a deploy my first FLASK application on PythonAnywhere.
Im not sure if this is the correct terminology but I have a folder as a module and there for I can't seem to find the correct way to deploy my application. I am not even sure where to start in resolving this issue. Any advice?
File and Folder Layout Snipped
my init.py code is:
import os
from flask import Flask
def create_app(test_config=None):
# create and configure the app
app = Flask(__name__, instance_relative_config=True)
app.config.from_mapping(
SECRET_KEY='secret',
DATABASE=os.path.join(app.instance_path, 'LAMA.sqlite'),
)
if test_config is None:
# load the instance config, if it exists, when not testing
app.config.from_pyfile('config.py', silent=True)
else:
# load the test config if passed in
app.config.from_mapping(test_config)
# ensure the instance folder exists
try:
os.makedirs(app.instance_path)
except OSError:
pass
# database
from . import db
db.init_app(app)
# authentication blueprint
from . import auth
app.register_blueprint(auth.bp)
# blog blueprint - the main index
# from . import blog
# app.register_blueprint(blog.bp)
# app.add_url_rule('/', endpoint='index')
# book blueprint
from . import book
app.register_blueprint(book.bp)
app.add_url_rule('/', endpoint='index')
return app
I have also followed the python debugging page where I have done the following:
>>> import LAMA
>>> print(LAMA)
<module 'LAMA' from '/home/ivanv257/LAMA_MAIN/LAMA/__init__.py'>
So at this stage in my WSGI configuration file I have:
import sys
path = '/home/ivanv257/LAMA_MAIN/LAMA/__init__.py'
if path not in sys.path:
sys.path.append(path)
from LAMA import app as application
I have also tried many other combinations such as
path = '/home/ivanv257/LAMA_MAIN/LAMA/'
from init import app as application
path = '/home/ivanv257/LAMA_MAIN/'
from init import app as application
path = '/home/ivanv257/LAMA_MAIN/'
from LAMA import app as application
my source code path is : /home/ivanv257/LAMA_MAIN/LAMA , although I have also tried different combinations such as /home/ivanv257/LAMA_MAIN/
ERROR DETAIL:
2018-12-08 10:05:32,028: Error running WSGI application
2018-12-08 10:05:32,030: ModuleNotFoundError: No module named 'LAMA'
2018-12-08 10:05:32,030: File "/var/www/ivanv257_pythonanywhere_com_wsgi.py", line 83, in <module>
2018-12-08 10:05:32,030: from LAMA import app as application # noqa

To solve my problem I changed the following (with some assistance) from lama import create_app:
import sys
path = '/home/ivanv257/LAMA_MAIN/LAMA'
if path not in sys.path:
sys.path.append(path)
from lama import create_app
application = create_app()
I also had to remove the from . to just imports
import db
db.init_app(app)
# authentication blueprint
import auth
app.register_blueprint(auth.bp)

You are close. To deploy your app, navigate to the webapps page on your user dashboard. If you have not done so already, click the "Add new a webapp" button and enter the desired name of the app. Then, on the same webapp page on the dashboard, scroll down to the "Code" section of the page. Click on the "source code" a href and add the absolute path (full path) to the lama_main directory storing your init.py file.
Next, click on "WSGI configuration file" link. In the WSGI configuration file for your app, set the correct path to the parent directory and import app from the init.py file:
import sys
# add your project directory to the sys.path
project_home = u'/home/your_user_name/lama_main'
if project_home not in sys.path:
sys.path = [project_home] + sys.path
# import flask app but need to call it "application" for WSGI to work
from init import app as application #note that the module being imported from must be the file with "app" defined.
Then, save the WSGI file and return to the webapp panel on your dashboard. Click the "Reload {your site name}" button. Now, you should be able to visit the site by clicking on the main link at the top of the page.

Django passenger_wsgi.py issue

I run Django on an A2 Hosting Linux shared hosting account. Previously, I was using an old version of Django and Python. I was also using FCGI. I recently changed to a new server and am now using Django 2.1.2, Python 3.6.0. and Phusion Passenger.
I can't get the passenger_wsgi.py to load my website and I can't find the error log to try and resolve the problem.
The .htaccess file is in the public_html directory.
My app files are stored in:
/home/username/example.com/
The folder/file structure in the example.com directory is:
-media
-mysite
--- init.py
--- settings.py
--- urls.py
--- wsgi.py
-public
-tmp
-manage.py
-passenger_wsgi.py
#
The contents of the passenger_wsgi.py file:
import os
import sys
import mysite.wsgi
application = mysite.wsgi.application
cwd = os.getcwd()
sys.path.append(cwd)
#sys.path.append(os.getcwd())
sys.path.append(cwd + '/mysite')
SCRIPT_NAME = os.getcwd()
class PassengerPathInfoFix(object):
"""
Sets PATH_INFO from REQUEST_URI since Passenger doesn't provide it.
"""
def __init__(self, app):
self.app = app
def __call__(self, environ, start_response):
from urllib.parse import unquote
environ['SCRIPT_NAME'] = SCRIPT_NAME
request_uri = unquote(environ['REQUEST_URI'])
script_name = unquote(environ.get('SCRIPT_NAME', ''))
offset = request_uri.startswith(script_name) and len(environ['SCRIPT_NAME']) or 0
environ['PATH_INFO'] = request_uri[offset:].split('?', 1)[0]
return self.app(environ, start_response)
application = PassengerPathInfoFix(application)
The site produces an error (image below). I've been trying to fix it all day by tinkering with the passenger_wsgi.py file.
I can't find the Passenger error log file. I don't have any experience with Passenger. I've run out of ideas of ways to try and resolve the issue.
When I run 'python passenger_wsgi.py' in SSH, no comments are produced.
If anyone could help me to either:
(1) Identify what is wrong with the passenger_wsgi.py content or
(2) How I can find the Passenger error log on a shared Linux hosting account.

Not sure it'll be possible to diagnose anything without logs, but I've had the similar issues on shared A2 Hosting finding logs. I was able to find some Passenger logs in the "Errors" section of cPanel.
On the cPanel home screen, scroll down to the "Metric" subheader. There will be a link for "Errors" in this section. You should be able to find critical Passenger logs there.

This structure of passenger_wsgi.py works for me on A2 Hosting:
import os
import sys
import MyApp.wsgi
SCRIPT_NAME = '/home/user/domain'
class PassengerPathInfoFix(object):
""" Sets PATH_INFO from REQUEST_URI because Passenger doesn't provide it. """
def __init__(self, app):
self.app = app
def __call__(self, environ, start_response):
from urllib.parse import unquote
environ['SCRIPT_NAME'] = SCRIPT_NAME
request_uri = unquote(environ['REQUEST_URI'])
script_name = unquote(environ.get('SCRIPT_NAME', ''))
offset = request_uri.startswith(script_name) and len(environ['SCRIPT_NAME']) or 0
environ['PATH_INFO'] = request_uri[offset:].split('?', 1)[0]
return self.app(environ, start_response)
application = MyApp.wsgi.application
application = PassengerPathInfoFix(application)

How can I make a Python package which serves a website with Flask to work with Apache?

I have written a Python package hwrt (see installation instructions if you want to try it) which serves a website when executed with
$ hwrt serve
2014-12-04 20:27:07,182 INFO * Running on http://127.0.0.1:5000/
2014-12-04 20:27:07,183 INFO * Restarting with reloader
I would like to let it run on http://www.pythonanywhere.com, but when I start it there I get
19:19 ~ $ hwrt serve
2014-12-04 19:19:59,282 INFO * Running on http://127.0.0.1:5000/
Traceback (most recent call last):
File "/home/MartinThoma/.local/bin/hwrt", line 108, in <module>
main(args)
File "/home/MartinThoma/.local/bin/hwrt", line 102, in main
serve.main()
File "/home/MartinThoma/.local/lib/python2.7/site-packages/hwrt/serve.py", line 95, in main
app.run()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 739, in run
run_simple(host, port, self, **options)
File "/usr/local/lib/python2.7/dist-packages/werkzeug/serving.py", line 613, in run_simple
test_socket.bind((hostname, port))
File "/usr/lib/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 98] Address already in use
I only found this in the documentation:
Flask
never use app.run(), it will break your webapp. Just import the
app into your wsgi file...
By searching for wsgi file, I found mod_wsgi (Apache). However, I don't understand how I can adjust my current minimalistic Flask application to work with that. Currently, the script behind hwrt serve is:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""Start a webserver which can record the data and work as a classifier."""
import pkg_resources
from flask import Flask, request, render_template
from flask_bootstrap import Bootstrap
import os
import json
# hwrt modules
import hwrt
import hwrt.utils as utils
def show_results(results, n=10):
"""Show the TOP n results of a classification."""
import nntoolkit
classification = nntoolkit.evaluate.show_results(results, n)
return "<pre>" + classification.replace("\n", "<br/>") + "</pre>"
# configuration
DEBUG = True
template_path = pkg_resources.resource_filename('hwrt', 'templates/')
# create our little application :)
app = Flask(__name__, template_folder=template_path)
Bootstrap(app)
app.config.from_object(__name__)
#app.route('/', methods=['POST', 'GET'])
def show_entries():
heartbeat = request.args.get('heartbeat', '')
return heartbeat
#app.route('/interactive', methods=['POST', 'GET'])
def interactive():
if request.method == 'POST':
raw_data_json = request.form['drawnJSON']
# TODO: Check recording
# TODO: Submit recorded json to database
# Classify
model_path = pkg_resources.resource_filename('hwrt', 'misc/')
model = os.path.join(model_path, "model.tar")
print(model)
results = utils.evaluate_model_single_recording(model, raw_data_json)
# Show classification page
page = show_results(results, n=10)
page += 'back'
return page
else:
# Page where the user can enter a recording
return render_template('canvas.html')
def get_json_result(results, n=10):
s = []
for res in results[:min(len(results), n)]:
s.append({res['semantics']: res['probability']})
return json.dumps(s)
#app.route('/worker', methods=['POST', 'GET'])
def worker():
# Test with
# wget --post-data 'classify=%5B%5B%7B%22x%22%3A334%2C%22y%22%3A407%2C%22time%22%3A1417704378719%7D%5D%5D' http://127.0.0.1:5000/worker
if request.method == 'POST':
raw_data_json = request.form['classify']
# TODO: Check recording
# TODO: Submit recorded json to database
# Classify
model_path = pkg_resources.resource_filename('hwrt', 'misc/')
model = os.path.join(model_path, "model.tar")
results = utils.evaluate_model_single_recording(model, raw_data_json)
return get_json_result(results, n=10)
else:
# Page where the user can enter a recording
return "Classification Worker (Version %s)" % hwrt.__version__
def get_parser():
"""Return the parser object for this script."""
from argparse import ArgumentParser, ArgumentDefaultsHelpFormatter
parser = ArgumentParser(description=__doc__,
formatter_class=ArgumentDefaultsHelpFormatter)
return parser
def main():
app.run()
if __name__ == '__main__':
main()

Ok, a not so non-sequitur answer to your question is around what mod_wsgi does to interface with your app. A typical flask app would look something like this:
from flask import Flask
app = Flask(__name__)
app.route("/")
def hello():
return "Holy moly that tunnel was bright.. said Bit to NIC"
if __name__ == "__main__":
app.run()
Unfortunately, Apache has no way to know what to do with this (though the app would run happily on its own). In order to get the app and Apache to play nice together we're going to use something called mod_wsgi. What Mod_WSGI does that's important to us, is that it provides a known interface (a file type called wsgi) that's going to wrap our application and initialize it so that we can serve it through Apache.
I'm going to assume you are using a python virtual environment, but if you aren't you can omit the step that deals with this in the instructions below. If you're curious why virtual environments are so great, feel free read about the python ecosystem.
Also - you can include an extra flag (assuming you are running wsgi as a daemon) to automatically reload the daemon whenever you touch or alter your wsgi file. This is quite useful during development and debugging so I'll include is below.
Anyway, let's get started. I'll break this down to steps below.
Configuring Apache for mod_wsgi
Enable mod_wsgi in Apache:
sudo apt-get install libapache2-mod-wsgi
Edit your /etc/apache2/sites-available/<yoursite>.conf.
<VirtualHost interface:port>
WSGIDaemonProcess yourapp user=someUser processes=2 threads=15
WSGIProcessGroup yourapp
# In this case / refers to whatever relative URL path hosts flask
WSGIScriptAlias / /absolute/path/to/yourapp.wsgi
<Directory /path/to/your/main/py/file/ >
# Use good judgement here when server hardening, this assumes dev env
Order allow,deny
Allow from all
Require all granted
#The below enables 'auto-reload' of WSGI
WSGIScriptReloading On
</Directory>
# If you want to serve static files as well and bypass flask in those cases
Alias /relative/url/to/static/content/
<Directory /absolute/path/to/static/root/directory/>
Order allow,deny
Allow from all
</Directory>
</VirtualHost>
Create your yourapp.wsgi file and put it in the appropriate place: Be wary of file permissions!
#!/usr/bin/python
import sys
import logging
# Activate virtual environment.
# If you are not using venv, skip this.
# But you really should be using it!
activate_this = "/path/to/venv/bin/activate_this.py"
execfile(activate_this, dict(__file__=activate_this))
# Handle logging
logging.basicConfig(stream=sys.stderr)
sys.path.insert(0, "/path/to/your/main/py/file/")
from YourMainPyFileName import app as application
application.secret_key = "your_secret_key"
Reload Apache and troubleshoot problems. I set this up probably every few weeks for a different project or idea I have and... I usually have to fix one thing or another when doing it from scratch. Don't despair though! Flask has great documentation on this.
Once you've done all this you should be at a place where flask runs all on its own. The sample flask app above is the actual code I use to verify everything works whenever I set this up.
This was left here in case it's some use, but is not really directly related to the question...
The answer here is to use x-send-file. This takes advantage of letting Apache do what it's good at (serving static content), while at the same time first letting flask (or other python framework) do it's work first. I do this often to let flask handle my auth layers in single page web apps and have so far been happy with the results.
Doing so requires two things:
First - Enable xsendfile on Apache2 sudo apt-get install libapache2-mod-xsendfile.
Second - Alter your apache2 configuration so allow x-send-file headers:
Alter your conf file in /etc/apache2/sites-available/<yoursite>.conf and add...
XSendFile On
XSendFilePath /path/to/static/directory
This can be entered top level within the <Virtualhost></Virtualhost> tags.
Don't forget to restart Apache sudo service apache2 restart.
Finally - Configure your flask app to use x-send-file in your app.py file:
app.user_x_sendfile = True
Note: Must be done after app initialization. Consequently can also be passed as an initialization parameter.
Flask has documentation on this (excerpt below):
use_x_sendfile
Enable this if you want to use the X-Sendfile feature. Keep in mind that the server has to support this. This only affects files sent with the send_file() method.
New in version 0.2.
This attribute can also be configured from the config with the USE_X_SENDFILE configuration key. Defaults to False.

I ran into a similar issue #moose was having. Getting connection refused and couldnt even telnet localhost 5000.
Turns out theres a ports.conf file i had to add Listen 5000
Happy days.

Django best way to support developer specific config/settings

Here is the method my company devised, just wondering if anyone has anything better:
In Settings.py (at the bottom)
#...
try:
if socket.gethostname() == 'testsite':
from myir.local.TEST_settings import *
elif socket.gethostname() == 'prod':
from myir.local.PROD_settings import *
else:
from myir.local.DEV_settings import *
try:
# dev settings - don't commit local_settings.py
from proj.local.local_settings import *
except:
print "no local dev settings found..."
pass # intentionally do nothing.
except ImportError:
pass
local_settings.py:
DEBUG = True
LOGGING = { .. } # i usually keep maximum aount of logging possible in my dev environment.
... other configs you might want to override.

Django provides the environment variable DJANGO_SETTINGS_MODULE to specify the settings module to use. You can specify settings.my_prod_module there which enables settings to be different on production. Locally you can specify a different value.
You can also specify the value in your WSGI file:
os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings.my_prod_module'

Flask: app.config settings from .env &. flaskenv in mod_wsgi

I have spent quite a while trying to figure out how to set .env and .flaskenv configuration values in my flask backend in Google Cloud Platform server. I am using apache2, mod_wsgi, Flask, Python 3.6 and SQLAlchemy. My backend works fine locally on my Mac using pure Flask.
Having python-dotenv installed, running the flask command will set environment variables defined in the files .env and .flaskenv. This, however, does not work with wsgi. The request from apache is redirected to execute my run.wsgi-file. There is no mechanism (that I have knowledge about) to set the environment variables defined in .env and .flaskenv.
The minimun requirement is to pass to the application information if test or development environment should be used. From there I could within init.py populate app.config values from an object. However, being somehow able to use config-values from .env and .flaskenv would be far better. I would really appreciate if somebody had any good ideas here - the best practice to set app.config values in wsgi environment.
There are two posts where this same problem has been presented - they really do not have a best practice how to tackle this challenge (and I am sure I am not the only one having a hard time with this):
Why can't Flask can't see my environment variables from Apache (mod_wsgi)?
Apache SetEnv not working as expected with mod_wsgi
My run.wsgi:
import sys
sys.path.append("/var/www/contacts-api/venv/lib/python3.6/site-packages")
sys.path.insert(0,"/var/www/contacts-api/")
from contacts import create_app
app = create_app('settings.py')
app.run()
[3]:Allows you to configure an application using pre-set methods.
from flask_appconfig import AppConfig
def create_app(configfile=None):
app = Flask('myapp')
AppConfig(app, configfile)
return app
The application returned by create_app will, in order:
Load default settings from a module called myapp.default_config, if it exists. (method described in http://flask.pocoo.org/docs/config/#configuring-from-files )
Load settings from a configuration file whose name is given in the environment variable MYAPP_CONFIG (see link from 1.).
Load json or string values directly from environment variables that start with a prefix of MYAPP_, i.e. setting MYAPP_SQLALCHEMY_ECHO=true will cause the setting of SQLALCHEMY_ECHO to be True.
Any of these behaviors can be altered or disabled by passing the appropriate options to the constructor or init_app().
[4]: Using “ENV-only”
If you only want to use the environment-parsing functions of Flask-AppConfig, the appropriate functions are exposed:
from flask_appconfig.heroku import from_heroku_envvars
from flask_appconfig.env import from_envvars
# from environment variables. note that you need to set the prefix, as
# no auto-detection can be done without an app object
from_envvars(app.config, prefix=app.name.upper() + '_')
# also possible: parse heroku configuration values
# any dict-like object will do as the first parameter
from_heroku_envvars(app.config)

After reading more about this and trying many different things. I reached to the conclusion that there is no reasonable way for configuring a Flask-application using .env- and .flaskenv -files. I ended up using a method presented in Configuration Handling which enables managing development/testing/production-environments in a reasonable manner:
app = Flask(__name__)
app.config.from_object('yourapplication.default_settings')
app.config.from_envvar('YOURAPPLICATION_SETTINGS')
My run.wsgi (being used in google cloud platform compute instance):
import sys
import os
from contacts import create_app
sys.path.append("/var/www/myapp/venv/lib/python3.6/site-packages")
sys.path.insert(0,"/var/www/myapp/")
os.environ['SETTINGS_PLATFORM_SPECIFIC'] = "/path/settings_platform_specific.py"
os.environ['CONFIG_ENVIRONMENT'] = 'DevelopmentConfig'
app = create_app(')
app.run()
Locally on my mac I use run.py (for flask run):
import os
from contacts import create_app
os.environ['SETTINGS_PLATFORM_SPECIFIC'] ="/path/settings_platform_specific.py"
os.environ['CONFIG_ENVIRONMENT'] = 'DevelopmentConfig'
if __name__ == '__main__':
app = create_app()
app.run()
For app creation init.py
def create_app():
app = Flask(__name__, template_folder='templates')
app.config.from_object(f'contacts.settings_common.{os.environ.get("CONFIG_ENVIRONMENT")}')
app.config.from_envvar('SETTINGS_PLATFORM_SPECIFIC')
db.init_app(app)
babel.init_app(app)
mail.init_app(app)
bcrypt.init_app(app)
app.register_blueprint(routes)
create_db(app)
return app
At this point it looks like this works out fine for my purposes. The most important thing is that I can easily manage different environments and deploy the backend service to google platform using git.

I was wrestling with the same conundrum, wanting to use the same .env file I was using with docker-compose while developing with flask run. I ended up using python-dotenv, like so:
In .env:
DEBUG=True
APPLICATION_ROOT=${PWD}
In config.py:
import os
from dotenv import load_dotenv
load_dotenv()
class Config(object):
SECRET_KEY = os.getenv('SECRET_KEY') or 'development-secret'
DEBUG = os.getenv("DEBUG") or False
APPLICATION_ROOT = os.getenv("APPLICATION_ROOT") or os.getcwd()
I haven't experimented with it yet, but I may also give flask-env a try in combination with this solution.

The easy to go will be using load_dotenv() and from_mapping
from flask import Flask
from dotenv import load_dotenv , dotenv_values
load_dotenv()
app = Flask(__name__)
config = dotenv_values()
app.config.from_mapping(config)