I'd like to break some specific views from a django app out into a separate NewRelic application. The NewRelic python docs implies that I can set request.environ['newrelic.app_name'] and it will use my new value instead of the default one, but it doesn't appear to work.
eg. in my wsgi file:
from django.core.handlers.wsgi import WSGIHandler
import newrelic.agent
application = WSGIHandler()
newrelic.agent.initialize('/path/to/newrelic.ini') # defines app_name='1st_app'
application = newrelic.agent.wsgi_application()(application)
in my Django view:
def some_view(request, *args, **kwargs):
request.environ['newrelic.app_name'] = '2nd_app'
...
return HttpResponse(...)
You can split these views out in mod_wsgi, but it must be done at the server level and not with Django. Use the SetEnv directive in your Apache config instead. Since you want to change the app_name based on request, you will need to use the Location directive to identify which URL should be re-named.
For example:
<Location /some/url>
SetEnv newrelic.app_name 2nd_app
</Location>
Related
How could I override or change the project's settings from an app? For example, I have a context processor function in my app, and I want to include it into a context_processors option of the TEMPLATE setting. How could I do that without changing project's settings.py file directly?
In the documentation i found, that they cannot be changed at runtime:
You shouldn’t alter settings in your applications at runtime. For
example, don’t do this in a view:
from django.conf import settings
settings.DEBUG = True # Don't do this!
The only place you should assign to settings is in a settings file.
So is it possible at all?
My simple Django app worked fine in debug mode (manage.py runserver), and works under WSGI+Apache on my dev box, but when I pushed to EC2 I began receiving intermittent (10-80% of the time) errors of Bad Request (400) for any URLs I try to view (whether in my app or in the Django admin.
Where can I find debug information about this? Nothing appears in /var/log/apache2/error.log, even with LogLevel=info. I have checked versions, logged the Request environment (cf. ModWSGI Debugging Tips) and see no major differences.
The one remaining thought I had is, I'm using the mod_wsgi from Ubuntu 12.04 (libapache2-mod-wsgi 3.3-4build1) which was built against Python 2.7.1; I have Python 2.7.3. And Django is 1.6, which is newer than the Ubuntu Precise version. I hesitate to start building packages from source since it's so hard to clean up and these seem like minor version changes...
Thank you for your help.
(For reference, here are the Apache config and WSGI apps)
Apache config (000-default)
<VirtualHost *:80>
ServerAdmin webmaster#localhost
DocumentRoot /var/www
WSGIScriptAlias /rz /usr/local/share/rz/rz.wsgi
...
rz.WSGI app
import os
import sys
import django.core.handlers.wsgi
import pprint
path = '/usr/local/share/rz'
if path not in sys.path:
sys.path.insert(0, path)
os.environ['DJANGO_SETTINGS_MODULE'] = 'rz.settings'
class LoggingMiddleware:
def __init__(self, application):
self.__application = application
def __call__(self, environ, start_response):
errors = environ['wsgi.errors']
pprint.pprint(('REQUEST', environ), stream=errors)
def _start_response(status, headers, *args):
pprint.pprint(('RESPONSE', status, headers), stream=errors)
return start_response(status, headers, *args)
return self.__application(environ, _start_response)
application = LoggingMiddleware(django.core.handlers.wsgi.WSGIHandler())
Add the ALLOWED_HOSTS setting to your settings.py like so...
ALLOWED_HOSTS = [
'.example.com', # Allow domain and subdomains
'.example.com.', # Also allow FQDN and subdomains
]
I had this same problem and found the answer here in the docs
update: django 1.6 docs are no longer online, I updated the link to go to the django 1.7 docs for ALLOWED_HOSTS setting.
If you've definitely set ALOWED_HOSTS - make sure your hostname doesn't contain underscores. It's technically illegal.
I had to print out various functions and it boiled down to this regex failing to detect a domain in django.http
host_validation_re = re.compile(r"^([a-z0-9.-]+|\[[a-f0-9]*:[a-f0-9:]+\])(:\d+)?$")
And indeed, my domain had an underscore in it.
This is not a solution, but for debugging purposes you might set the ALLOWED_HOSTS setting in your settings.py like this
ALLOWED_HOSTS = ['*']
It should definitely work. If not, at least you will know the problem isn't Django denying access to the given url.
Is it possible to dynamically load Django apps at runtime? Usually, apps are loaded at initialization, using the INSTALLED_APPS tuple in settings.py. However, is it possible to load additional apps at runtime? I am encountering this issue in different situations. One situation, for example, arises during testing, when I would like to dynamically load or unload apps.
In order to make the problem more concrete, imagine I have a directory called apps where I put my apps and I would like to automatically install any new app that goes in there without manually editing the settings.py.
This is easy enough. Following the example code in
Django: Dynamically add apps as plugin, building urls and other settings automatically
we put the following code in settings.py to could loop over the names of all sub-directories in the app directory and increment the INSTALLED_APPS tuple in settings.py like this:
APPS_DIR = '/path_to/apps/'
for item in os.listdir(APPS_DIR):
if os.path.isdir(os.path.join(APPS_DIR, item)):
app_name = 'apps.%s' % item
if app_name not in INSTALLED_APPS:
INSTALLED_APPS += (app_name, )
After that, if I was in a Django shell, I could something like
from django.conf import settings
and the apps would be listed in settings.INSTALLED_APPS. And if I did
from django.core import management
management.call_command('syncdb', interactive=False)
that would create the necessary DB tables for the apps.
However, if I were to now add some more apps to the apps/ directory, without re-starting, these would not be listed in settings.INSTALLED_APPS, and so a subsequent call to the syncdb would have no effect.
What I would like to know is if there is something I could do --- without restarting --- to reload the settings and load/install new apps.
I have tried to directly import my settings.py, i.e.
from myproject import settings
and then reload that settings using the python builtin after any app directory changes. Although settings.INSTALLED_APPS is now changed to include the newly added apps, this ultimately makes no difference. For example,
from django.db import models
models.get_apps()
shows only the original apps in apps and not the newly added ones and likewise
management.call_command('syncdb', interactive=False)
will not see the newly added apps.
As I stated above, I am thinking about this situation particularly in the context of testings where I dynamically would add or remove apps.
Ps. I working with Django 1.6, but on the advice of #RickyA, I see that there are some substantial changes to Django's treatment of applications in 1.7
https://docs.djangoproject.com/en/1.7/ref/applications/
I'm still not sure what this might mean for the problem I am facing.
Update for Django 1.8 on how to load an app that is not loaded yet
from collections import OrderedDict
from django.apps import apps
from django.conf import settings
from django.core import management
new_app_name = "my_new_app"
settings.INSTALLED_APPS += (new_app_name, )
# To load the new app let's reset app_configs, the dictionary
# with the configuration of loaded apps
apps.app_configs = OrderedDict()
# set ready to false so that populate will work
apps.ready = False
# re-initialize them all; is there a way to add just one without reloading them all?
apps.populate(settings.INSTALLED_APPS)
# now I can generate the migrations for the new app
management.call_command('makemigrations', new_app_name, interactive=False)
# and migrate it
management.call_command('migrate', new_app_name, interactive=False)
With Django 2.2 this work for me
from collections import OrderedDict
from django.apps import apps
from django.conf import settings
from django.core import management
new_app_name = "my_new_app"
settings.INSTALLED_APPS += (new_app_name, )
apps.app_configs = OrderedDict()
apps.apps_ready = apps.models_ready = apps.loading = apps.ready = False
apps.clear_cache()
apps.populate(settings.INSTALLED_APPS)
management.call_command('makemigrations', new_app_name, interactive=False)
management.call_command('migrate', new_app_name, interactive=False)
To answer my own question...
While I do not have a completely general solution to this problem, I do have one that is sufficient for the purposes of dynamically loading apps during testing.
The basic solution is simple, and I found it at a wee little bixly blog.
Continuing with my example above, if I was in a django shell and wanted to add and load some new apps that were added to my apps directory, I could do
import os
from django.conf import settings
from django.db.models import loading
from django.core import management
APPS_DIR = '/path_to/apps/'
for item in os.listdir(APPS_DIR):
if os.path.isdir(os.path.join(APPS_DIR, item)):
app_name = 'apps.%s' % item
if app_name not in settings.INSTALLED_APPS:
settings.INSTALLED_APPS += (app_name, )
and then
loading.cache.loaded = False
management.call_command('syncdb', interactive=False)
It is possible to dynamically load and unload applications in tests in Django >= 1.7 (and also in the current 4.1) by a override_settings() decorator:
#override_settings(INSTALLED_APPS=[...]) # added or removed some apps
class MyTest(TestCase):
# some tests with these apps
def test_foo(self):
pass
It has been possible since September 2014, not before the question.
Many other topics from the question are solved by django.apps also in Django >= 1.7. Generally: Dynamic configuration at startup is easy in the current Django. Dynamic loading and unloading after startup can't be recommended in production, even that it could work eventually.
Yes! Anything (or almost everything) in Python is possible. You should use os.walk() in order to get all folders, subfolders and files within your apps path in order to get all of your apps including the nested ones.
def get_installed_apps():
from os import walk, chdir, getcwd
previous_path = getcwd()
master = []
APPS_ROOT_PATH = '/my/project/apps/folder'
chdir(APPS_ROOT_PATH)
for root, directories, files in walk(top=getcwd(), topdown=False):
for file in files:
if 'apps.py' in file:
app_path = f"{root.replace(BASE_DIR + '/', '').replace('/', '.')}.apps"
print(app_path)
master.append(app_path)
chdir(previous_path)
return master
print(get_installed_apps())
I have an application that is constructed as follows:
app = Flask(__name__)
app.wsgi_app = DispatcherMiddleware(frontend.create_app(), {
'/api': api.create_app()
})
app.config['DATABASE'] = db
I want to access the same database in both the frontend and api apps, but when I run something like current_app.config['DATABASE'] in Blueprints registered to api, that raises KeyError: 'DATABASE'. Is it possible to inherit configurations so that tests, etc. need only modify the top-level abstraction? Thanks.
Simply change your create_app methods on frontend and api to take a configuration dictionary and use flask.Config to create a configuration with all the properties of app.config (loadable from environmental variables, etc.):
from flask import Config
from werkzeug.wsgi import DispatcherMiddlware
config = Config()
config.from_envvar("YOUR_PROGRAM_CONFIG")
app = DispatcherMiddlware(frontend.create_app(config), {
'/api': api.create_app(config)
})
Then you can merge the provided config with each of your app's configurations:
def create_app(config):
app = Flask(__name__)
app.config.update(config)
# Potentially, load another config
app.config.from_envvar("YOUR_PROGRAM_CONFIG_FRONTEND", silent=True)
return app
Note that there is no need to create an app to route to other apps - the dispatcher middleware can already do that for you.
Also note that if you are already using blueprints it may make more sense to simply ship the blueprints and compose them on a single application, rather than using dispatcher middleware (depending, of course on how complex your apps are).
I am trying to run a django app and a webapp2 app together in one python interpreter. I'm using werkzeug for that as described here.
Here's my sample code.
from werkzeug.wsgi import DispatcherMiddleware
from django_app import application as djangoapp
from webapp2_app import application as webapp2app
application = DispatcherMiddleware(djangoapp, {
'/backend': webapp2app
})
After doing this, I would expect all requests to /backend should be treated by the webapp2 app as /. But it treats the requests as /backend. This work fines with other WSGI apps using django or flask. The problem only appears with webapp2 apps. Does anyone have any suggestions how to overcome this? Is there any other way I can achieve my purpose without using werkzeug for serving multiple WSGI apps under one domain?
DispatcherMiddleware fabricates environments for your apps and especially SCRIPT_NAME. Django can deal with it with configuration varibale FORCE_SCRIPT_NAME = '' (docs).
With Webapp2 it's slightly more complicated. You can create subclass of webapp2.WSGIApplication and override __call__() method and force SCRIPT_NAME to desired value. So in your webapp2_app.py it could be like this
import webapp2
class WSGIApp(webapp2.WSGIApplication):
def __call__(self, environ, start_response):
environ['SCRIPT_NAME'] = ''
return super(WSGIApp, self).__call__(environ, start_response)
# app = WSGIApp(...)