I would like to run some environment checks when my django process starts and die noisily in the case of an error. I'm thinking things like the database has an incorrect encoding or the machine has a python version we don't support.
I'd rather our team be faced with a fatal error that they have to fix, rather than be able to ignore it.
I'm Ok with writing these checks but I'm curious about where the best place to put them is. How do I get them to execute as part of django's startup process? I thought there might be a signal I could listen too, but I can't find a relevant one in the docs.
If you don't want to use settings module, then try project's __init__.py.
If you want to check that the system is correctly installed, I think that you should write your own admin command and run it as post-installation check.
I think that it doesn't worth to check if the python version is correctly installed too often especially if you are installing the django app on shared-host. My app is hosted at alwaysdata and they restart the FastCgi process every hour. These checks can have an impact on the application response time.
We use the top-level urls.py for this.
I would put them in settings.py. In the past, I have put system checks like this:
try:
from local_settings import *
except ImportError:
print "Missing %s" % os.path.join(PROJECT_ROOT, "local_settings.py")
if DEBUG:
for p in [PROJECT_ROOT, MEDIA_ROOT, THEME_DIR, ADMIN_MEDIA_ROOT] + list(TEMPLATE_DIRS):
p = os.path.normpath(p)
if not os.path.exists(p):
print "Missing path: %s" % p
I have tested all three __init__.py Settings.py and urls.py methods and this is what I have found.
When code is run from either __init__.py or Settings.py, the start up functions are run twice upon web server start up; when the start up functions are run from urls.py the code is run once, however it is run only upon the first request made to the we server leading to a potentially long wait for the first user to visit your site.
It is standard practise to call a 'warming' page when bringing large web applications back online so I don't see that calling a start up function from a CLEARLY identified location in the urls.py should be a problem.
Most of the answers here are extremely old, so I assume that's why Django's checks framework isn't mentioned.
It's been a part of django since at least v2 (django==3.1 at the time of writing).
That's probably the right place for most of the checks you're requiring.
I'd still consider using settings, as mentioned by other answers, for checks where settings contents are required but which also need to be run prior to readying the apps.
You can put it in settings.py as mentioned by others, but having code in the settings is not ideal. There is also the option of adding a handler for django.db.models.signals.class_prepared that does the desired start up checks after a specific model class is prepared.
Related
It was all ok till suddenly I got the above error while implementing DjangoObjectPermissions on my APIs.
Before this It was working ok even my production environment it is working fine. I am seeing this error on my local environment only.
according to this answer, the error would go away, but I need to know why?
Please let me know what information should I add to this post.
Following are the related packages installed.
Django==1.10
django-allauth==0.29.0
django-angular==0.8.3
django-debug-toolbar==1.6
django-debug-toolbar-request-history==0.0.3
django-debug-toolbar-template-profiler==1.0.1
django-debug-toolbar-template-timings==0.7
djangorestframework==3.5.3
Add try/except statement to enforce_csrf()
Note that this error originates from rest_framework/authentication.py, within class SessionAuthentication, method enforce_csrf(). The enforce_csrf() method iinitiates variable, check = CSRFCheck(), and the next line says "check.process_request(request).
If using an IDE, you'll quickly notice that CSRFCheck() doesn't have this attribute/method thus the error. Many developers will quickly tell you to upgrade to django >= 1.11.6 but you are bound to run into the same error.
For that matter, am using django 1.11.6 and rest_framework 3.9.1.
So try this remedy, it worked for me. Use the try/except statement.
Go within the rest_framework/authentication.py, below, check = CSRFCheck(),
add this...
def enforce_csrf(self, request):
...
try:
check.process_request(request)
except:
pass
What the lines say is this:
After declaring the "check" variable, call (try) this method "process_request()" and if it does not work (except), just pass. You achieve two things with this: One, you literally leave the rest_framework source code compatible (given that upgrading to further versions of django "might" solve this), and two, you get working code (nice!!, especially if working Agile)
From Django docs:
You shouldn’t alter settings in your applications at runtime. For
example, don’t do this in a view:
from django.conf import settings
settings.DEBUG = True # Don't do this!
The only place you should assign to settings is in a settings file.
I've noticed that Django testing code does alter settings. Why is it ok to do it there?
Is it ok to change settings?
Short answer:
No, unless you do it during the startup.
Long answer:
Django docs are correct, you should not modify settings at runtime. This means, no settings modifications after the app has been started, like changing configuration in views.py, serializers.py, models.py or other modules you add during the development. But it is ok to modify settings if they depend on local variables if you do it at startup and you are fully aware of what happens.
Can you modify settings while testing?
Yes, if you think you need it. Feel free to rely on override_settings to change settings value for testing purposes in the unit tests, see example of usage here
Also, everything that this decorator does - is overriding settings with provided values and restoring settings value after test has passed (decorated function executed).
Why Django does modify them while testing the code.
From what I see, they change settings only for testing purposes and the only thing they do - adding a local host to allowed host so they can test the code using a local domain. Examples like that seem pretty reasonable for me as change is done only once and during unit tests set up. Imagine having overrride_settings call every time, that would be monstrous.
General recommendation.
Try not to, there is no need to modify settings and if there is - think about it, maybe settings is not the right place for a mutable setting?
If you want to modify settings at runtime - please be aware that settings might be cached somewhere, copied and accessed all over the place - this is a plenty of space for new bugs. There is nothing bad in it, except having an unexpected behavior of the system due to an old/new value of the modified setting.
Hope this makes sense.
The answer is in the wording:
You shouldn’t alter settings in your applications at runtime.
Unit test code is not part of your application, so that statement doesn't apply to unit tests.
Why is it ok to do it there?
As per above, it is perfectly OK to override settings during tests, provided you do it in a localised manner (because tests are sometimes run in a multi-threaded manner).
Here is how they recommend doing it:
from django.test import TestCase
class LoginTestCase(TestCase):
def test_login(self):
# First check for the default behavior
response = self.client.get('/sekrit/')
self.assertRedirects(response, '/accounts/login/?next=/sekrit/')
# Then override the LOGIN_URL setting
with self.settings(LOGIN_URL='/other/login/'):
response = self.client.get('/sekrit/')
self.assertRedirects(response, '/other/login/?next=/sekrit/')
See docs:
https://docs.djangoproject.com/en/2.2/topics/testing/tools/#django.test.SimpleTestCase.settings
Changing the settings during tests is perfectly normal, expected, supported behavior. This is because you want to verify that your code works with lots of different settings. It's so normal in fact that the built-in method to do so has such a simple name it's a bit confusing to find in the docs:
e.g.
class QueueTests(TestCase):
def test_both_modes(self):
with self.settings(QUEUE_MODE='fast'):
self.assertTrue(run_a_test())
assert settings.QUEUE_MODE == 'fast'
with self.settings(QUEUE_MODE='slow'):
self.assertTrue(run_a_test())
assert settings.QUEUE_MODE == 'slow'
I have been told that doing this would be a not-very-good practice (it is present in the main answer of Python pattern for sharing configuration throughout application though):
configfile.py
SOUNDENABLED = 1
FILEPATH = 'D:\\TEMP\\hello.txt'
main.py
import configfile
if configfile.SOUNDENABLED == 1:
....
f = open(configfile.FILEPATH, 'a')
...
This is confirmed by the fact that many people use INI files for local configuration with ConfigParser module or iniparse or other similar modules.
Question: Why would using an INI file for local configuration + an INI parser Python module be better than just importing a configfile.py file containing the right config values as constants?
The only concern here is that a .py can have arbitrary Python code, so it has a potential to break your program in arbitrary ways.
If you can trust your users to use it responsibly, there's nothing wrong with this setup. If fact, at one of my previous occupations, we were doing just that, without any problems that I'm aware of. Just on the contrary: it allowed users to eliminate duplication by autogenerating repetitive parts and importing other config files.
Another concern is if you have many files, the configuration ones are better be separated from regular code ones, so users know which files they are supposed to be able to edit (the above link addresses this, too).
Importing a module executes any code that it contains. Nothing restricts your configfile.py to containing only definitions. Down the line, this is a recipe for security concerns and obscure errors. Also, you are bound to Python's module search path for finding the configuration file. What if you want to place the configuration file somewhere else, or if there is a name conflict?
This is a perfectly acceptable practice. Some examples of well-known projects using this method are Django and gunicorn.
It could be better for some reasons
The only extension that config file could have is py.
You cannot distribute your program with configs in separate directory unless you put an __init__.py into this directory
Nasty users of your program can put any python script in config and do bad things.
For example, the YouCompleteMe autocompletion engine stores config in python module, .ycm_extra_conf.py. By default, each time config is imported, it asks you, whether you sure that the file is safe to be executed.
How would you change configuration without restarting your app?
Generally, allowing execution of code that came from somewhere outside is a vulnerability, that could lead to very serious consequences.
However, if you don't care about these, for example, you are developing web application that executes only on your server, this is an acceptable practice to put configuration into python module. Django does so.
There is a pretty strange issue happening on my production server that doesn't come up in UAT or my development environment. Stack setup is Ubuntu 14LTS, Apache2, Python 2.7,Celery, RabbitMQ, Django 1.6.
How it works is that there is an async task that is called whenever an excel report needs to be generated. Depending on the type of company, a different report is created and emailed to the logged in user. The functions to create the different reports are all placed in a file called report_lib.py and they are not importing from any other app. The code goes like this:
if policies.carrier == "class_one":
report_lib.create_remittance_class_one_report(company_id)
else:
report_lib.create_generic_remittance_report(company_id)
Get this, the first conditional works no problem. The second conditional works when I don't have a company type set(works as expected), but for any other class it fails with the following error:
[2014-10-27 17:56:55,271: WARNING/Worker-1] There was an error: 'module' object has no attribute 'create_generic_remittance_report'
To make things more interesting I removed the conditional and defaulted to the code just calling the "create_generic_remittance_report" function for all cases, which works without complaining about that module.
I'm 99% sure that there are no-circular references. Oh, and the library being referenced is under the only app being used in the django project. Could this be a compiler caching issue?
Has anyone come across a similar issue using the same setup?
Please help!
Found the issue, the function being referenced was indented an extra tab. I guess since celery only calls it, it came back with that error.
Totally new to Openshift and has been following various stepbystep guides. Able to get django 1.6, Python 2.7 and Mezzanine 3.0.9 up with the application working - partially. For some reason, the template is not loaded, both if the template is part of a HTML include tag or a part of view.py.
When tailing access-log, cannot see errors or anything to go by. The settings.py has
PROJECT_ROOT = os.path.dirname(os.path.abspath(__file__))
PROJECT_DIRNAME = PROJECT_ROOT.split(os.sep)[-1]
TEMPLATE_DIRS = (os.path.join(PROJECT_ROOT, "templates"),)
It seems to not be able to find the path of the template files but don't know why as the value of TEMPLATE_DIRS seems to be correct when checking it. Everything is working ok on my local machine but not on Openshift. Any pointers are much appreciated as have been googling and search around for a few days and still get no where.
Thank you.
EDIT:
Decide to turn on DEBUG mode and that is a lot clearer to investigate. Turns out that without providing absolute module name when importing a method the application just fail but this is not the case on local machine.
e.g. instead of providing
from projectname.appname.view import some_function
I was putting
from appname.view import some_function
Silly me. That teach me a good few days lesson!!!durr!!
Problem is resolved by providing a full module name to the import statement.
e.g.
from projectname.appname.view import some_function
I had, see below, and that caused the issue.
from appname.view import some_function
Once a full module name is provided, it all works perfectly.
Thank you.