I working on my little package, and I came across with the weird werkzeug problem.
Problem:
When I tried to run my application in debug mode, and in order to run my application I use the command python3 -m my_app --start
I got the error ModuleNotFoundError: No module named '__main__.helpers'; '__main__' is not a package
the files are very simple
app.py
class Application:
"""Main class"""
def __init__(self):
self.reader = ApplicationReader()
self.app = Flask(self.reader.name)
def __start_server(self):
"""Custom method, for starting flask application"""
self.app.run(debug=True)
def run(self):
"""Run the server"""
return self.__start_server()
# Instance of Application class
application = Application()
def start_app():
"""Start the mock server
"""
return application.run()
__main__.py
def cli(run, init, start):
if run:
print('Your API is running...')
if init:
create_init_file()
if start:
start_app()
if __name__ == '__main__':
cli()
And seems like the bug can't be fixed, GitHub issue
So, maybe someone have a solution for this case? Thanks.
I created a different file for the start_app function because I found that in debug mode, the import could not be realized.
start_app.py (next to app.py)
import sys, os
sys.path.append(os.path.join(os.path.dirname(__file__)))
print("sys.path", sys.path) # use this to check if the last entry has the right folder
from app import start_app
start_app()
Now, you can run it with python -m start_app.py in debug mode.
Related
I'm trying to run Flask from an imported module (creating a wrapper using decorators).
Basically I have:
app.py:
import mywrapper
#mywrapper.entrypoint
def test():
print("HEYO!")
mywrapper.py
from flask import Flask
ENTRYPOINT = None
app = Flask(__name__)
#app.route("/")
def listen():
"""Start the model API service"""
ENTRYPOINT()
def entrypoint(f):
global ENTRYPOINT
ENTRYPOINT = f
return f
FLASK_APP=app
Running python -m flask, however, results in:
flask.cli.NoAppException: Failed to find Flask application or factory in module "app". Use "FLASK_APP=app:name to specify one.
Is there any trick to getting Flask to run like this? Or is it just not possible? The purpose of this is to abstract Flask away in this situation.
In my head flask should try to import mywrapper.py, which imports app.py which should generate the app and route, yet this doesn't seem to be what occurs.
Any help would be appreciated.
So I've since learnt that Flask searches only in the chosen module's namespace for a variable containing a Flask object.
There may be a smart way to avoid this limitation, but I instead decided that it was more sensible to instead just wrap the Flask class itself. If people want direct Flask functionality, I don't really care in this situation, so the only real limitation I have from this is some function names are off limits.
Basically:
wrapper.py:
class Wrapper(Flask):
def __init__(self, name):
super().__init__(name)
self.entrypoint_func = None
#self.route("/")
def listen():
return self.entrypoint_func()
def entrypoint(self, f):
assert self.entrypoint_func is None, "Entrypoint can only be set once"
self.entrypoint_func = f
return f
and app.py:
from mywrapper import Wrapper
app = Wrapper(__name__)
#app.entrypoint
def test():
print("HEYO!")
return "SUCCESS"
This is still abstracted enough that I am happy with the results.
I'm having issues with deploying a Flask Application with WSGI and PythonAnywhere. I assume I have done something wrong but I wasn't able to find exactly what I did do wrong.
Here's my WSGI config file
import sys
# add your project directory to the sys.path
project_home = u'/home/{redacted}/src'
if project_home not in sys.path:
sys.path = [project_home] + sys.path
from start import run as application
Here's start.py
from bin import app
run = app.register().run()
Although if I try it this way it still does not work
from bin import app
run = app.register()
And here's what I'm doing in bin/main.py
class Application():
# Some required code redacted, although not necessary for this example
def __init__(self):
self.app = Flask(__name__, instance_relative_config=True)
def register(self):
""" Starts Flask App """
try:
self.configure_app()
self.register_blueprints()
except Exception as e:
print(f"Error configuring Application: {e}")
return self.app
app = Application()
Just in case you're interested, here's how it's imported through bin/__ init __.py
from .main import app
I do understand that the method in which I am doing this is over complicating it a little bit, and that I could get it working with a more normal method. Although I want it done this way and I'm fairly sure it can work this way, because the example I've shown 100% works on my machine.
Help is very appreciated. I am 100% aware I am doing something wrong here I just need help figuring out what it is.
I’m relatively new to python and am looking for a pythonic way to handle this practice.
I’ve inherited a fairly trivial Python 2.7 Flask app that runs under uwsgi that I want to add some unit tests to. It does some initialization at indentation level 0 that is required when it’s running in uwsgi but needs to be skipped when under test.
I’m given to understand that often python apps use the
if __name__ == '__main__':
pattern to isolate code that should run when the script is run on its own and should not run when it’s imported. In this case, however, both when the script is run under uwsgi and when the script is imported into the unit tests, __name__ is the same; the name of the script, so I can’t use that to differentiate between uwsgi and unit-testing environments.
This sample code illustrates what I'm working with.
In the Flask application (flask_app.py):
import logging
import bcrypt
from flask import Flask, jsonify, abort, make_response, request
from ConfigParser import SafeConfigParser
# some initialization that only makes sense when running from the uwsgi context on the server
PARSER = SafeConfigParser()
PARSER.read('config.ini')
LOG_FILE = PARSER.get('General', 'logfile')
APP = Flask(__name__)
#APP.route('/', methods=['GET'])
def index
...
#APP.route('/<p1>/<p2>', methods=['PUT'])
def put(p1, p2):
...
if __name__ == '__main__':
APP.run(debug = True, host='0.0.0.0')
In the unit tests (tests.py):
import os
import unittest
from flask import json
from flask_app import APP
class FlaskAppTestCase(unittest.TestCase):
def setUp(self):
self.APP = APP.test_client()
def test_GET(self):
resp = self.APP.get('/')
assert 'Some Html' in resp.data
def test_PUT(self):
resp = self.APP.put('/1/2')
assert 'Got 1, 2' in resp.data
if __name__ == '__main__':
unittest.main()
What I was thinking of doing was to move the initialization so that it only runs when flask_app is being executed by uwsgi and not when it's running via tests.py, perhaps by checking name and determining which path to execute based on that, but when I examine the output of print(name) either when running flask_app under uwsgi or by executing tests.py the output is "flask_app", so I can't seem to use that as a discriminator.
Is there an idiomatic way in python to handle this?
As it turns out the Python module for uWSGI actually offers a mechanism to determine if the app is being run under uWSGI. The uwsgi module is available for import if you are in a uWSGI context, so I ended up checking whether i could import that module and only executing the initialization code if I could.
# detect if we're running under uWSGI; only init if we are, not if we're testing
try:
import uwsgi
IN_UWSGI = True
except ImportError:
IN_UWSGI = False
Then wrap the init code with
if IN_UWSGI:
This seems much more reliable then checking the module name of the module that's doing the import, which was the only other thing I could think of to do.
In a real Pyramid app it does not work per docs http://docs.pylonsproject.org/projects/pyramid//en/latest/narr/testing.html :
class FunctionalTests(unittest.TestCase):
def setUp(self):
from myapp import main
app = main({})
Exception:
Traceback (most recent call last):
File "C:\projects\myapp\tests\model\task_dispatcher_integration_test.py", line 35, in setUp
app = main({})
File "C:\projects\myapp\myapp\__init__.py", line 207, in main
engine = engine_from_config(settings, 'sqlalchemy.')
File "C:\projects\myapp\ve\lib\site-packages\sqlalchemy\engine\__init__.py", line 407, in engine_from_config
url = options.pop('url')
KeyError: 'url'
The reason is trivial: an empty dictionary is passed to main, while it seems that while running real app (from __init__.py) it gets settings pre-filled with values from [app:main] section of development.ini / production.ini:
settings {'ldap_port': '4032', 'sqlalchemy.url': 'postgresql://.....}
Is there some way of reconstructing settings easily from an .ini file for functional testing?
pyramid.paster.get_appsettings is the only thing you need:
from pyramid.paster import get_appsettings
settings = get_appsettings('test.ini', name='main')
app = main(settings)
That test.ini can include all the settings of another .ini file easily like this:
[app:main]
use = config:development.ini#main
and then you only need to override those keys that change (I guess you'd want to rather test against a separate DB):
[app:main]
use = config:development.ini#main
sqlalchemy.uri = postgresql://....
In case anyone else doesn't get #antti-haapala's answer right away:
Create a test.ini filled with:
[app:main]
use = config:development.ini#main
(Actually this step is not necessary. You could also keep your development.ini and use it instead of test.ini in the following code. A separate test.ini might however be useful if you want separate settings for testing)
In your tests.py add:
from pyramid.paster import get_appsettings
settings = get_appsettings('test.ini', name='main')
and replace
app = TestApp(main({}))
with
app = TestApp(main(global_config = None, **settings))
Relevant for this answer was the following comment: Pyramid fails to start when webtest and sqlalchemy are used together
Actually, you don't need import get_appsettings, just add the
parameters like this:
class FunctionalTests(unittest.TestCase):
def setUp(self):
from myapp import main
settings = {'sqlalchemy.url': 'sqlite://'}
app = main({}, **settings)
here is the source: functional test, it is in the second block code, line 31.
Yes there is, though the easy is a subject to debate.
I am using the following py.test test fixture to make --ini option passed to the tests. However this approach is limited to py.test test runner, as other test runner do not have such flexibility.
Also my test.ini has special settings like disabling outgoing mail and instead printing it out to terminal and test accessible backlog.
#pytest.fixture(scope='session')
def ini_settings(request):
"""Load INI settings for test run from py.test command line.
Example:
py.test yourpackage -s --ini=test.ini
:return: Adictionary representing the key/value pairs in an ``app`` section within the file represented by ``config_uri``
"""
if not getattr(request.config.option, "ini", None):
raise RuntimeError("You need to give --ini test.ini command line option to py.test to find our test settings")
# Unrelated, but if you need to poke standard Python ConfigParser do it here
# from websauna.utils.configincluder import monkey_patch_paster_config_parser
# monkey_patch_paster_config_parser()
config_uri = os.path.abspath(request.config.option.ini)
setup_logging(config_uri)
config = get_appsettings(config_uri)
# To pass the config filename itself forward
config["_ini_file"] = config_uri
return config
Then I can set up app (note that here pyramid.paster.bootstrap parses the config file again:
#pytest.fixture(scope='session')
def app(request, ini_settings, **settings_overrides):
"""Initialize WSGI application from INI file given on the command line.
TODO: This can be run only once per testing session, as SQLAlchemy does some stupid shit on import, leaks globals and if you run it again it doesn't work. E.g. trying to manually call ``app()`` twice::
Class <class 'websauna.referral.models.ReferralProgram'> already has been instrumented declaratively
:param settings_overrides: Override specific settings for the test case.
:return: WSGI application instance as created by ``Initializer.make_wsgi_app()``.
"""
if not getattr(request.config.option, "ini", None):
raise RuntimeError("You need to give --ini test.ini command line option to py.test to find our test settings")
data = bootstrap(ini_settings["_ini_file"])
return data["app"]
Furthermore setting up a functional test server:
import threading
import time
from wsgiref.simple_server import make_server
from urllib.parse import urlparse
from pyramid.paster import bootstrap
import pytest
from webtest import TestApp
from backports import typing
#: The URL where WSGI server is run from where Selenium browser loads the pages
HOST_BASE = "http://localhost:8521"
class ServerThread(threading.Thread):
""" Run WSGI server on a background thread.
Pass in WSGI app object and serve pages from it for Selenium browser.
"""
def __init__(self, app, hostbase=HOST_BASE):
threading.Thread.__init__(self)
self.app = app
self.srv = None
self.daemon = True
self.hostbase = hostbase
def run(self):
"""Open WSGI server to listen to HOST_BASE address
"""
parts = urlparse(self.hostbase)
domain, port = parts.netloc.split(":")
self.srv = make_server(domain, int(port), self.app)
try:
self.srv.serve_forever()
except Exception as e:
# We are a background thread so we have problems to interrupt tests in the case of error
import traceback
traceback.print_exc()
# Failed to start
self.srv = None
def quit(self):
"""Stop test webserver."""
if self.srv:
self.srv.shutdown()
#pytest.fixture(scope='session')
def web_server(request, app) -> str:
"""py.test fixture to create a WSGI web server for functional tests.
:param app: py.test fixture for constructing a WSGI application
:return: localhost URL where the web server is running.
"""
server = ServerThread(app)
server.start()
# Wait randomish time to allows SocketServer to initialize itself.
# TODO: Replace this with proper event telling the server is up.
time.sleep(0.1)
assert server.srv is not None, "Could not start the test web server"
host_base = HOST_BASE
def teardown():
server.quit()
request.addfinalizer(teardown)
return host_base
Imagine I have a big CLI application with many different commands (think, for example image-magick).
I wanted to organize this application into modules and etc. So, there would be a master click.group somewhere:
#main.py file
#click.group()
def my_app():
pass
if __name__ == "__main__":
my_app()
that can be imported in each module that define a command:
from main import my_app
# command_x.py
#my_app.command()
def command_x():
pass
The problem is that I run into a circular import problem, since the main.py file knows nothing about command_x.py and I would have to import it before calling the main section.
This happens in Flask too and is usually dealt with the app factory pattern. Usually you would have the app being created before the views:
app = Flask("my_app")
#my_app.route("/")
def view_x():
pass
if __name__ == "__main__":
app.run()
In the app factory pattern you postpone the "registration" of the blueprints:
# blueprints.py
blueprint = Blueprint(yaddayadda)
#blueprint.route("/")
def view_x():
pass
And make a factory that knows how to build the app and register the blueprints:
#app_factory.py
from blueprints import view_x
def create_app():
app = Flask()
view_x.init_app(app)
return app
And you can then create a script to run the app:
#main.py
from app_factory import create_app
if __name__ == "__main__":
app = create_app()
app.run()
Can a similar pattern be used with Click? Could I just create a "click app" (maybe extending click.Group) where I register the "controllers" which are the individual commands?
Maybe late, but I was also searching for a solution to put commands to separate modules. Simply use a decorator to inject commands from modules:
#main.py file
import click
import commands
def lazyloader(f):
# f is an instance of click.Group
f.add_command(commands.my_command)
return f
#lazyloader
#click.group()
def my_app():
pass
if __name__ == "__main__":
my_app()
The separated command can use the usual decorators from click.
#commands.py
import click
#click.command()
def my_command():
pass
Ok, so I thought a little and it seems that the following could work. It's probably not a final solution but it seems to be an initial step.
I can extend the MultiCommand class:
# my_click_classes.py
import click
class ClickApp(click.MultiCommand):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.commands = {}
def add_command(self, command_name, command):
self.commands.update({command_name: command})
def list_commands(self, ctx):
return [name for name, _ in self.commands.items()]
def get_command(self, ctx, name):
return self.commands.get(name)
And the Command class:
class MyCommand(click.Command):
def init_app(self, app):
return app.add_command(self.name, self)
def mycommand(*args, **kwargs):
return click.command(cls=MyCommand)
This allows you to have the commands defined in separated modules:
# commands.py
from my_click_classes import command
#command
def run():
print("run!!!")
#command
def walk():
print("walk...")
and the "app" in a separated module:
from my_click_classes import ClickApp
from commands import run, walk
app = ClickApp()
run.init_app(app)
walk.init_app(app)
if __name__ == '__main__':
app()
Or even use the "app factory" pattern.
It maybe not a definitive solution though. If you guys can see any way to improve it, please let me know.