I have the following code at the end of my Django settings:
if not TESTING:
# Don't use local settings for tests, so that tests are always reproducible.
try:
from .local_settings import *
except ImportError:
pass
local_settings.py contains all the external dependencies URLs used by my Django application, such as database server URL, email server URL and external APIs URLs. Currently the only external dependency my test suite uses is a local test database; everything else is mocked.
However, now I'd like to add some tests that validate responses from the external APIs I use, so that I can detect quickly when an external API changes without prior notice. I'd like to add a --external-deps command line argument to "./manage.py test" and only run tests that depend on external APIs if this flag is enabled.
I know that I can process arguments passed to that command by overriding the add_arguments() method of the DiscoverRunner class, as described in Django manage.py : Is it possible to pass command line argument (for unit testing) , but my conditional Django settings loading are run before that, so the following won't work:
if not TESTING or TEST_EXTERNAL_DEPS:
# Don't use local settings for tests, so that tests are always reproducible.
try:
from .local_settings import *
except ImportError:
pass
Is there a way to achieve what I want?
I integrated Flask-Migrate into my project. When I'm using development mode (FLASK_ENV='development') I would normally call flask db migrate to apply changes to SQLite database. But, in testing mode (FLASK_ENV='testing') I'm using internal memory storage (sqlite:///:memory:) and it has no sense to call db migrate because it will end up throwing error. Is there some way to create "pre_execute" hook in Flask CLI to check which ENV is used before executing command? So for example if FLASK_ENV is set to testing than calling flask db init will result in aborting execution of command. I've tried something like this but it didn't work:
#click.group(cls=FlaskGroup, create_app=create_app)
def cli():
'''
Main entry point.
'''
if app.config.ENV == ENV.TESTING:
print('Running in TESTING mode...Aborting!')
sys.exit(1)
Question: How I can abort execution of cli command under certain FLASK_ENV setting?
Edit: I'm loading FLASK_ENV value from .env file.
Okay, maybe at first I tried to solve problem with wrong approach, but I finally found way to deal with an error mentioned in my question. Because I load value of FLASK_ENV from file I need manually change it every time I want to switch environment. So what I did is I modified my test CLI command to set value of FLASK_ENV to testing every time before executing pytest:
#click.command()
def test():
'''
Run tests.
'''
os.environ['FLASK_ENV'] = ENV.TESTING
pytest.main(['--rootdir', './tests'])
Now even if FLASK_ENV set to development in .env file I still can run tests in testing mode without changing value in the file.
I have some configuration in a json file and on the database and I want to load those configuration on Django startup (Apache server startup).. I will be using those global variable within all the application.
For Example: External server connection api or number of instances.
What is the best way to define the global variables. I want to load the json file when server starts and use the variable value util server stop. ?
It sounds like the thing you're probably looking for is environment variables - you can always use a small script to set the environment variables from the JSON that you have at present.
Setting these in your .bashrc file or, more preferably a virtualenv will let you:
Take sensitive settings, like SECRET_KEY out of version control.
Offer database settings, either by supplying them as a DB URL or as seperate environment variables.
Set both Django settings and other useful variables outside of the immediate Django project.
The django-environ docs have a useful tutorial on how to set it up. The Django Cookie-Cutter project makes extensive use of Environment Variables (including DB and mailserver settings), and is a great place to pick up hints and approaches.
I just started using Fabric to better control the specific settings for test and deployment environments, and I'm trying to get an idea of the best approach to swapping configurations.
Let's say I have a module in my application that defines a simple database connection and some constants for authentication by default:
host = 'db.host.com'
user = 'someuser'
passw = 'somepass'
db = 'somedb'
class DB():
def __init__(self,host=host,user=user,passw=passw,db=db,cursor='DictCursor'):
#make a database connection here and all that jazz
Before I found fabric, I would use the getfqdn() function from the socket library to check the domain name of the host the system was being pushed to and then conditionalize the authentication credentials.
if getfqdn() == 'test.somedomain.com':
host = 'db.host.com'
user = 'someuser'
passw = 'somepass'
db = 'somedb'
elif getfqdn() == 'test.someotherdomain.com':
host = 'db.other.com'
user = 'otherguy'
passw = 'otherpass'
db = 'somedb'
This, for obvious reasons, is really not that great. What I would like to know is what's the smartest way of adapting something like this in Fabric, so that when the project gets pushed to a certain test/deployment server, these values are changed at post-push.
I can think of a few approaches just from looking through the docs. Should I have a file that just defines the constants that Fabric could output to using the shell commands based off what the deployment was, and then the file defining the database handler could import them? Does it makes sense to run open and write from within the fabfile like this? I assumed I'd also have to .gitignore these kinds of files so they don't get committed into the repo and just rely on Fabric to deploy them.
I plan on adapting whatever approach is the best suggested to all the configuration settings that I currently either swap using getfqdn or adjust manually. Thanks!
You can do all of that off of the env.host and then use something like the contrib template function to render the conf file and push it up. But templates are best in these instances (see: puppet and other config managers as well)
How can I be certain that my application is running on development server or not? I suppose I could check value of settings.DEBUG and assume if DEBUG is True then it's running on development server, but I'd prefer to know for sure than relying on convention.
I put the following in my settings.py to distinguish between the standard dev server and production:
import sys
RUNNING_DEVSERVER = (len(sys.argv) > 1 and sys.argv[1] == 'runserver')
This also relies on convention, however.
(Amended per Daniel Magnusson's comment)
server = request.META.get('wsgi.file_wrapper', None)
if server is not None and server.__module__ == 'django.core.servers.basehttp':
print('inside dev')
Of course, wsgi.file_wrapper might be set on META, and have a class from a module named django.core.servers.basehttp by extreme coincidence on another server environment, but I hope this will have you covered.
By the way, I discovered this by making a syntatically invalid template while running on the development server, and searched for interesting stuff on the Traceback and the Request information sections, so I'm just editing my answer to corroborate with Nate's ideas.
Usually this works:
import sys
if 'runserver' in sys.argv:
# you use runserver
Typically I set a variable called environment and set it to "DEVELOPMENT", "STAGING" or "PRODUCTION". Within the settings file I can then add basic logic to change which settings are being used, based on environment.
EDIT: Additionally, you can simply use this logic to include different settings.py files that override the base settings. For example:
if environment == "DEBUG":
from debugsettings import *
Relying on settings.DEBUG is the most elegant way AFAICS as it is also used in Django code base on occasion.
I suppose what you really want is a way to set that flag automatically without needing you update it manually everytime you upload the project to production servers.
For that I check the path of settings.py (in settings.py) to determine what server the project is running on:
if __file__ == "path to settings.py in my development machine":
DEBUG = True
elif __file__ in [paths of production servers]:
DEBUG = False
else:
raise WhereTheHellIsThisServedException()
Mind you, you might also prefer doing this check with environment variables as #Soviut suggests. But as someone developing on Windows and serving on Linux checking the file paths was plain easier than going with environment variables.
I came across this problem just now, and ended up writing a solution similar to Aryeh Leib Taurog's. My main difference is that I want to differentiate between a production and dev environments when running the server, but also when running some one-off scripts for my app (which I run like DJANGO_SETTINGS_MODULE=settings python [the script] ). In this case, simply looking at whether argv[1] == runserver isn't enough. So what I came up with is to pass an extra command-line argument when I run the devserver, and also when I run my scripts, and just look for that argument in settings.py. So the code looks like this:
if '--in-development' in sys.argv:
## YES! we're in dev
pass
else:
## Nope, this is prod
pass
then, running the django server becomes
python manage.py runserver [whatever options you want] --in-development
and running my scripts is as easy as
DJANGO_SETTINGS_MODULE=settings python [myscript] --in-development
Just make sure the extra argument you pass along doens't conflict with anything django (in reality I use my app's name as part of the argument).
I think this is pretty decent, as it lets me control exactly when my server and scripts will behave as prod or dev, and I'm not relying on anyone else's conventions, other than my own.
EDIT: manage.py complains if you pass unrecognized options, so you need to change the code in settings.py to be something like
if sys.argv[0] == 'manage.py' or '--in-development' in sys.argv:
# ...
pass
Although this works, I recognize it's not the most elegant of solutions...
If you want to switch your settings files automatically dependent on the runtime environment
you could just use something that differs in environ, e.g.
from os import environ
if environ.get('_', ''):
print "This is dev - not Apache mod_wsgi"
You can determine whether you're running under WSGI (mod_wsgi, gunicorn, waitress, etc.) vs. manage.py (runserver, test, migrate, etc.) or anything else:
import sys
WSGI = 'django.core.wsgi' in sys.modules
settings.DEBUG could be True and running under Apache or some other non-development server. It will still run. As far as I can tell, there is nothing in the run-time environment short of examining the pid and comparing to pids in the OS that will give you this information.
I use:
DEV_SERVERS = [
'mymachine.local',
]
DEVELOPMENT = platform.node() in DEV_SERVERS
which requires paying attention to what is returned by .node() on your machines. It's important that the default be non-development so that you don't accidentally expose sensitive development information.
You could also look into more complicated ways of uniquely identifying computers.
One difference between the development and deployment environment is going to be the server that it’s running on. What exactly is different will depend on your dev and deployment environments.
Knowing your own dev and deploy environments, the HTTP request variables could be used to distinguish between the two. Look at request variables like request.META.HTTP_HOST, request.META.SERVER_NAME and request.META.SERVER_PORT and compare them in the two environments.
I bet you’ll find something quite obvious that’s different and can be used to detect your development environment. Do the test in settings.py and set a variable that you can use elsewhere.
Inspired by Aryeh's answer, the trick I devised for my own use is to just look for the name of my management script in sys.argv[0]:
USING_DEV_SERVER = "pulpdist/manage_site.py" in sys.argv[0]
(My use case is to automatically enable Django native authentication when running the test server - when running under Apache, even on development servers, all authentication for my current project is handled via Kerberos)
You could check request.META["SERVER_SOFTWARE"] value:
dev_servers = ["WSGIServer", "Werkzeug"]
if any(server in request.META["SERVER_SOFTWARE"] for server in dev_servers):
print("is local")
Simple you may check the path you work on server. Something like:
import os
SERVER = True if os.path.exists('/var/www/your_project') else False