I am having a strange issue, hopefully someone has encountered it before.
I have a custom script located at my_project/my_app/scripts/custom_script.py -- My desired use case is to feed this script a filepath and have it process the file and load/modify some data within my database.
Here is a snippet from that script:
import django
def setup_django(env):
if env == 'dev':
settings = "my_project.my_project.dev_settings"
elif env == 'stg':
settings = "my_project.my_project.staging_settings"
elif env == 'prod':
settings = "my_project.my_project.prod_settings"
else:
raise ValueError("Invalid choice for --env argument: {}".format(env))
os.environ.setdefault('DJANGO_SETTINGS_MODULE', settings)
django.setup()
def main():
env = 'dev'
setup_django(env)
# Do stuff with my script
When I run the above from within the same virtualenv as my project, I get the error ModuleNotFoundError: No module named 'my_app'
Inside the settings file:
...
INSTALLED_APPS = [
'my_app',
'django.contrib.admin',
'django.contrib.auth',
...
]
...
It seems that, when I call django.setup(), it goes and parses my my dev_settings file and finds my_app in the INSTALLED_APPS list, then tries to import it directly (e.g. import my_app). This will never work, because my_app is a sub-module of my_project (should be import my_project.my_app). Here is a sample of the directory structure:
my_project
├── my_project
│ ├── __init__.py
│ ├── dev_settings.py
│ ├── prod_settings.py
│ ├── staging_settings.py
│ ├── urls.py
│ └── wsgi.py
├── my_app
│ ├── __init__.py
│ ├── admin.py
│ ├── apps.py
│ ├── models.py
│ ├── scripts
│ │ ├── __init__.py
│ │ ├── **custom_script.py**
│ │ └── utils.py
│ ├── tests.py
│ ├── urls.py
│ └── views.py
├── __init__.py
└── manage.py
This settings file seems to work fine with manage.py, both in runserver and shell (can find the my_app application just fine). However inside the custom_script.py all seems lost.
Any ideas?
Use management commands to add custom scripts to your app so that all django configs get loaded up correctly.
In your my_app folder, create folder structure below
management/
__init__.py
commands/
__init__.py
my_command.py
In my_command.py
from django.core.management import BaseCommand
class Command(BaseCommand):
def handle(self, *args, **options):
print('running custom command')
on your shell
python manage.py my_command
Related
I have a question regarding pytest and django 3.1. I have an app structure like that:
main_app
├── config
│ └── settings
│ └── base.py / local.py / prod.py
├── main_app
│ ├── sub_apps
│ │ └── models.py tests.py apps.py views.py etc...
│ ├── templates
│ │ ├── account
│ │ ├── sub_apps
│ │ ├── pages
│ │ ├── userpreferences
│ │ └── users
│ ├── userpreferences
│ │ └── models.py tests.py apps.py views.py etc...
│ ├── users
│ │ └── tests --> test_models.py test_views.py test_forms.py etc...
│ └── utils
└── requirements
Following the structure of pydanny/cookiecutter.
With this cookiecutter, the preferred test method is via pytest. But pytest is giving me a bunch of headaches:
In my settings, INSTALLED_APPS I register sub_apps within the
LOCAL_APPS =["sub_app_1", "sub_app_2", ...]
When I start django, everything is fine. But if I want to run tests, pytest is complaining heavily, it can't import the sub_app modules.
I suspect mixed up relative and absolute import paths but am not seeing how to solve this atm.
As a background: I use a separate userpreferences model which is imported in the settings via
LOCAL_APPS = ["main_app.userpreferences.apps.UserpreferencesConfig"]
In the apps.py I have to define the app name as
name = "main_app.userpreferences"
Otherwise I get an Runtime error stating the Model class does'nt declare an explicit app_label / is not installed in INSTALLED_APPS.
While django runs just fine with these imports, pytest exits before running tests with:
ModuleNotFoundError: No module named "sub_app_1"
I tried to change every import of modules in the sub_apps to e.g.:
from .sub_app_1 import models --> from main_app.sub_app_1 import models
but then django won't run anymore while the tests seem to run (pytest is starting and telling me 5/7 tests passed, while also occasionally printing import errors)
How do I resolve this?
I have a Django project in which certain environment variables are set in manage.py which later serve as the values defined in settings.py. Therefore, in order to run pytest-django, I'd like to run manage.py first.
I'm trying to follow the instructions in https://pytest-django.readthedocs.io/en/latest/faq.html#how-can-i-use-manage-py-test-with-pytest-django, but I'm running into an unexpected error. I have the following directory structure:
.
├── lucy
│ ├── settings
│ ├── base.py
│ ├── development.py
│ ├── production.py
│ └── staging.py
│ ├── staticfiles
│ ├── urls.py
│ └── wsgi.py
├── lucy_web
│ ├── __init__.py
│ ├── actions.py
│ ├── admin
│ ├── apps.py
│ ├── fixtures
│ ├── forms
│ ├── lib
│ ├── management
│ ├── migrations
│ ├── models
│ ├── runner.py
│ ├── serializers.py
│ ├── static
│ ├── templates
│ ├── templatetags
│ ├── tests
│ ├── urls.py
│ └── views
├── manage.py
├── pytest.ini
The contents of runner.py are taken exactly from the FAQ:
class PytestTestRunner(object):
"""Runs pytest to discover and run tests."""
def __init__(self, verbosity=1, failfast=False, keepdb=False, **kwargs):
self.verbosity = verbosity
self.failfast = failfast
self.keepdb = keepdb
def run_tests(self, test_labels):
"""Run pytest and return the exitcode.
It translates some of Django's test command option to pytest's.
"""
import pytest
argv = []
if self.verbosity == 0:
argv.append('--quiet')
if self.verbosity == 2:
argv.append('--verbose')
if self.verbosity == 3:
argv.append('-vv')
if self.failfast:
argv.append('--exitfirst')
if self.keepdb:
argv.append('--reuse-db')
argv.extend(test_labels)
return pytest.main(argv)
Finally, in lucy/settings/base.py I add the following line:
TEST_RUNNER = 'lucy_web.runner.PytestTestRunner'
Finally, the pytest.ini file is as in the example in the docs:
# -- FILE: pytest.ini (or tox.ini)
[pytest]
DJANGO_SETTINGS_MODULE = lucy.settings.production
# -- recommended but optional:
python_files = tests.py test_*.py *_tests.py
The problem is that pytest doesn't seem able to find the tests. If I run the command
python manage.py test lucy_web.tests
I get
(venv) Kurts-MacBook-Pro:lucy-web kurtpeek$ python manage.py test lucy_web.tests
============================================ test session starts ============================================
platform darwin -- Python 3.6.4, pytest-3.3.2, py-1.5.2, pluggy-0.6.0
rootdir: /Users/kurtpeek/Documents/Dev/lucy/lucy-web, inifile: pytest.ini
======================================= no tests ran in 0.00 seconds ========================================
ERROR: file not found: lucy_web.tests
However, if I comment out the TEST_RUNNER line in base.py and run the same command, the tests run successfully:
(venv) Kurts-MacBook-Pro:lucy-web kurtpeek$ python manage.py test lucy_web.tests
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
....E.F................
----------------------------------------------------------------------
Ran 23 tests in 15.974s
FAILED (failures=1, errors=1)
Destroying test database for alias 'default'...
What am I doing wrong here? Do I need to put runner.py in a different location?
Maybe it's because of __init__.py
I experienced a similar situation when using pytest. I deleted __init__.py in tests dir and it tested well.
It's little bit different situation, but I hope this helps.
links that I got help.
https://docs.pytest.org/en/latest/pythonpath.html
`py.test` and `__init__.py` files
Django version 1.9.7.
My current project structure is:
vehicles/
├── etl
│ ├── etl
│ ├── manage.py
│ ├── pipeline
│ └── bku
└── web
├── db.sqlite3
├── manage.py
├── profiles
├── projects
├── reverse
├── static
├── templates
├── bku
│ ├── admin.py
│ ├── admin.pyc
│ ├── apps.py
│ ├── migrations
│ ├── models.py
│ ├── static
│ ├── templates
│ ├── tests.py
│ ├── urls.py
│ ├── views.py
│ └── views.pyc
└── rocket
├── celery.py
├── __init__.py
├── settings
│ ├── base.py
│ ├── dev.py
│ ├── __init__.py
│ ├── local.py
│ ├── production.py
│ ├── test.py
├── urls.py
├── wsgi.py
Now I want to use Celery in the bku Django app. But when I run the worker celery -A rocket worker -l info I get the following error django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.. I have the SECRET_KEY defined and I didn't have this error before trying Celery.
How can I run the worker?
rocket/celery.py
from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'rocket.settings')
app = Celery('rocket')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()
#app.task(bind=True)
def debug_task(self):
print('Request: {0!r}'.format(self.request))
rocket/init.py
from __future__ import absolute_import, unicode_literals
from .celery import app as celery_app
__all__ = ['bku']
The error message is a bit misleading—usually when you see an ImproperlyConfigured exception like that it means that Django can't find your settings file.
In your case you're setting the DJANGO_SETTINGS_MODULE environment variable to rocket.settings, but from your directory structure it looks like it should instead be something like rocket.settings.production.
Here is my project folder structure.
<pre>
front
├── __init__.py
├── __init__.pyc
├── manage.py
├── middleware.py
├── settings.py
├── news
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── admin.py
│ ├── admin_views.py
│ ├── models.py
│ ├── search_indexes.py
│ ├── search_sites.py
│ ├── utils.py
│ ├── views.py
</pre>
After I run
./manage.py runserver
then I visit http://127.0.0.1:8000/
It gives me error:
No module named front
This is caused by the following line in file views.py under news folder.
from front import settings
So the front folder is one level up to the views.py file. How I import the settings from one level up folder?
Thanks!
Either way this is wrong, whenever you're importing your own settings you should be importing from django.conf
from django.conf import settings
This isn't a module, its an object that does magic to import any settings you have set in your DJANGO_SETTINGS_MODULE
From the docs:
Also note that your code should not import from either global_settings or your own settings file. django.conf.settings abstracts the concepts of default settings and site-specific settings; it presents a single interface. It also decouples the code that uses settings from the location of your settings.
I am trying to study how to use alembic in flask, I want to import a method in flask app:
tree .
.
├── README.md
├── alembic
│ ├── README
│ ├── env.py
│ ├── env.pyc
│ ├── script.py.mako
│ └── versions
│ ├── 8f167daabe6_create_account_table.py
│ └── 8f167daabe6_create_account_table.pyc
├── alembic.ini
├── app
│ ├── __init__.py
│ ├── main
│ │ ├── __init__.py
│ │ ├── errors.py
│ │ ├── forms.py
│ │ └── views.py
│ ├── models.py
│ └── templates
│ ├── 404.html
│ ├── 500.html
│ ├── base.html
│ ├── index.html
│ └── user.html
├── config.py
├── data.sqlite
├── manage.py
└── requirements.txt
in app/__init__.py:
def create_app(config_name):
app = Flask(__name__)
I want to import create_app in env.py:
from app import create_app
but the error shows as below when I run the command alembic upgrade head:
File "alembic/env.py", line 5, in <module>
from app import create_app
ImportError: No module named app
Any idea for this?
I guess you are trying to run
python env.py
In this case, your app directory is not in PYTHONPATH.
solution 1
Run the app from parent dir:
python alembic/env.py
solution 2
Set the PYTHONPATH before running the app
PYTHONPATH=/path/to/parent/dir python env.py
edit
I read about alembic. As #mrorno said, just set the PYTHONPATH before running alembic:
PYTHONPATH=. alembic upgrade head
alembic just tries to load your env.py source code. It's not in your package, so it can't access your app module.
Use solution 2 #tomasz-jakub-rup suggested, you can execute like
$ PYTHONPATH=. alembic upgrade head
and you should get your result.
Create file .env and insert PYTHONPATH=.