I frequently run into practical problems when I want to test a library I develop which is a Django app. Django apps can be developed independently, like DRF, etc.
For testing, you either need a Django project, or at least a settings.py file or the like, see here.
But I don't get how to do that properly for a "standalone" library. I need to generate migrations for that library (if it contains models) - so I need a manage.py file to invoke manage.py makemigrations, I need to check if the lib is integrating into the project properly (in my case apps are full-fledged plugins, I am using GDAPS).
What is the best approach here?
should I create a separate "test project" in a separate repo that uses my library and tests it?
should I create a project within my library's tests directory and use that? CAVE python paths...
should I not use the project at all and mimick everything - how?
Please give me at least a hint in some direction.
Was googling the same question, and found that the "Using the Django test runner to test reusable applications" section of the Advanced testing topics of the Django documentation gives a suitable common practice. It's a mixture of all three your approaches.
Basically you should create a tests package next to the application code, with a test_settings.py containing all the settings your library needs, and at least set the SECRET_KEY and INSTALLED_APPS variables.
SECRET_KEY = 'fake-key'
INSTALLED_APPS = [
"tests", "your-library"
]
Where "your-library" is the name of the libray you need to test.
In my case, I had to also add 'django.contrib.auth', 'django.contrib.contenttypes', as they where needed by my library.
Then you put all your tests into that package, eventual addictional models in a models.py file, and run them with a runtests.py that lives on the root of your repo:
#!/usr/bin/env python
import os
import sys
import django
from django.conf import settings
from django.test.utils import get_runner
if __name__ == "__main__":
os.environ['DJANGO_SETTINGS_MODULE'] = 'tests.test_settings'
django.setup()
TestRunner = get_runner(settings)
test_runner = TestRunner()
failures = test_runner.run_tests(["tests"])
sys.exit(bool(failures))
It will use the test_settings.py as its settings file (os.environ['DJANGO_SETTINGS_MODULE'] = 'tests.test_settings'), and run all the tests contained in the tests package.
Related
I have a django 1.11 project with some rest_framework related apps. Writing tests for new app, I have suddenly gotten the issue
'RuntimeError: Model class core.myApp.models.query_record
doesn't declare an explicit app_label and isn't in an
application in INSTALLED_APPS'
I do have this listed in installed_apps, and in the end, the reason I have this issue is because I have an __init.py__ file in the top level of the project that loads some config for some related celery tasks.
I'm unaware why I haven't seen this issue in other app tests, as there is nothing particularly special about this app or it's model. But, this is causing all tests to fail for this app.
So, my question is, is there a way I can run these unit tests and ignore the projects top level __init.py__ ? Or maybe I should ask, is there a non-hacky way to do it?
the project level __init.py__:
from __future__ import absolute_import
from .celeryapp import app as celery_app
All other app init.py files are empty.
A problem might be, that .celeryapp is trying to import some models that aren't loaded yet.
You can try to add a AppConfig to core.myApp and load/import your celery app in it's ready() method. See the Django docs for more information Django docs for more information
It turned out, in the end this was simply because of how I was running tests
I was running tests like this
./manage.py test myApp --pattern=*.py
The pure wildcard was causing import issues. I should have done this:
./manage.py test myApp --pattern=prefix_*.py
D'oh.
This is an unusual situation - most Django users want Django's test runner to find all of their tests.
I'm working on a python library with several test suites that are run with different configurations, so I don't want the discovery to find and run tests from the wrong configuration. How do I disable discovery entirely and rely on the pre-1.6 behavior of only running the tests for apps explicitly declared in INSTALLED_APPS?
My library structure:
library/ # django app used by others
tests/ # custom test suites here
core/ # tests of core functionality
custom/ # tests of a custom feature requiring separate config
contrib/ # tests for assorted contrib features, also requiring separate config
manage_core.py # separate manage.py files for each "project"
manage_custom.py # these specify settings file to use.
manage_contrib.py
settings.py # base settings for all tests
settings_core.py # settings for 'core' tests including unique INSTALLED_APPS
settings_custom.py # settings for 'custom' tests; different INSTALLED_APPS
settings_contrib.py # settings for 'contrib' tests; different INSTALLED_APPS
The problem is that this command, which should only run tests for the 'contrib' test suite, is also finding and running tests for 'core':
./manage_contrib.py test contrib.tests
It's missing from the Django docs, but the command-line has an option, found via ./manage.py help test:
-t TOP_LEVEL, --top-level-directory TOP_LEVEL
Top level of project for unittest discovery.
Confusingly, specifying the module to test doesn't appear to prevent test discovery, but specifying a sub-directory does, like this:
./manage_contrib.py test contrib.tests -t ./contrib/
That appears to prevent the discovery of tests located outside of contrib.
Hmm I'm unfortunately not aware of a settings parameter that might let you tell unittest to only run from individual apps (a-la "settings.TEST_DIRECTORIES=settings.INSTALLED_APPS") but if you're able to give your tests a unique naming convention, you could use the --pattern= option when running the test suite.
For example, if you have
/myapp/tests/test_a_models.py
/myapp/tests/test_b_models.py
You could only run a with ./manage.py test --pattern='*_a_*' and then run b with ./manage.py test --pattern='*_b_*'
Definitely not ideal, but might get the job done depending on how much flexibility you have with the test naming conventions in your own app.
I am trying to execute an external snippet for debugging and terminal-like purposes in the environment the Django console uses so it can connect to the db, etc.
Basically, I am just using it for the same reason one would fiddle with the console but I am using longer snippets to output some formatted information so it is handy to have that code in an actual file manipulated with an IDE.
An answer said you could do that by executing python manage.py shell < snippet.py but I did not see a successfull result. And although no errors are reported, I am not getting the excepted output, but only a series of >>> prompts.
So how can I do this?
By the way, I am using PyCharm, in case this IDE has a shorthand way of doing this or any special tool.
I would say creating a new Custom management command is the best way to achieve this goal.
But you can run your script in a django environment. I use this sometimes to run a oneoff script or some simple tests.
You have to set the environment variable DJANGO_SETTINGS_MODULE to your settings module and then you have to call django.setup()
I copied these lines from the manage.py script, you have to set the correct settings module!
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings.local")
django.setup()
Here is a simple template script which I use sometimes:
# -*- coding: utf-8 -*-
import os
import django
# you have to set the correct path to you settings module
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings.local")
django.setup()
from project.apps.bla.models import MyModel
def run():
# do the work
m = MyModel.objects.get(pk=1)
if __name__ == '__main__':
run()
It is important to note that all project imports must be placed after calling django.setup().
Many times you need scripts to play with db or more. But you need to the the django way i.e. the ORM and play with every thing that your project has.
You can checkout https://github.com/django-extensions/django-extensions
You create a scripts folder in your project home and write down a method run . the app provide you a way to run those scripts easily.
How to do it.
Step 1 - Install the package.
Step 2 - Create a scripts folder with init.py in project root, and add 'django_extensions' in your applications
Step 3- Create a file send_email_to_users.py in scripts folder. Simple Example.
from project.models import MyModel
from django.contrib.auth.models import User
from project.utils import send_email
def run():
results = MyModel.objects.all()
for res in results:
send_mail(res.email)
Now from command line you can run
python manage.py runscript send_mail_to_users
If its just a one off script,
import django
django.setup()
from myapp.models.import MyModel
You need to have your environmental variable set up, so its easiest to run it from the IDE (make it part of the same project, right click on teh file and there should be a run option).
If you are looking for something to run on a production environment I would create a management command as suggested by DanEEStar.
I'm starting on my first large django project and have realized that the default directory structure isn't going to work well.
I saw this question and answer and decided to implement something similar.
Large Django application layout
What do I need to do to make something like this work? How can I change where django looks for apps/modules?
Python works automatically with deep directory structures. That's porbably you didn't find any instructions on how to do it.Here are some instructions on how to get classes and models to work.
If you want to have a module in folder yourproject/apps/firstapp you can just add it to INSTALLED_APPS by adding line 'apps.firstapp',. You will have to add a __init__.py file to each of the directories so that they are recognized as python packages.
When you import classes you can simply use from yourproject.apps.firstapp.filename import yourclass.
You should also make sure all template directories are listed in TEMPLATE_DIRS.
I have two methods to handle this; one for production and one for development.
In development:
Append the path to your apps in your settings.py. In my case, I have my main project in ~/hg/django/project/ and my apps are in ~/hg/django/apps/. So I use:
if DEVEL:
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', 'apps'))
I also use the production method on my development box. This method has a knock on effect of letting me roll back to the most recent production version by simply commenting out the line path insertion in the settings.py.
In production:
I install the apps using distutils so that I can control deployment per server, and multiple projects running on each server can all access them. You can read about the distutils setup scripts here. Then, to install, you simply:
./setup.py install
if you add:
import os
PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__))
to your settings .py and the following to your manage.py:
sys.path.insert(0, join(settings.PROJECT_ROOT, "apps"))
sys.path.insert(0, join(settings.PROJECT_ROOT, "lib"))
then you can include them in your installed apps just as you would if they were in your project root:
INSTALLED_APPS = (
#local apps
'myapp',
#local libs
'piston_dev',
)
this allows you a bit more freedom to move apps around without having to redeclare imports and such.
I have problems with importing correctly a module on appengine. My app generally uses django with app-engine-patch, but this part is task queues using only the webapp framework.
I need to import django settings for the app to work properly.
My script starts with:
import os
import sys
sys.path.append('common/')
# Force Django to reload its settings.
from django.conf import settings
settings._target = None
# Must set this env var before importing any part of Django
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
I always get this error, or something related:
<type 'exceptions.ImportError'>: No module named ragendja.settings_pre
because the settings.py file starts with
from ragendja.settings_pre import *
I think I need to add ragendja to sys.path again but I had several tries that didn't work.
Here is my directory:
project/
app.yaml
setting.py
common/
appenginepatch/
ragendja/
setting_pre.py
myapp/
script.py
Is it only a sys.path problem and how do I need to modify it with the correct syntax?
Thanks
App engine patch manipulates sys.path internally. Background tasks bypass that code, so your path will not be ready for Django calls. You have two choices:
Fix the paths manually. The app engine documentation (see the sub-section called "Handling import path manipulation") suggests factoring the path manipulation code into a module that can be imported by your task script.
Eliminate dependencies on django code, if possible. If you can write your task to be pure python and/or google api calls, you're good to go. In your case, this might mean refactoring your settings code.
Why not:
sys.path.append('common/appenginepatch')
since the ragendja is in this directory?