Here is what I did:
I wrapped my 12 Django apps in an apps folder in my Django project.
I added sys.path.append(os.path.join(BASE_DIR, 'apps')) in settings.py (before INSTALLED_APPS declaration) sothat I don't need to specify the apps folder when importing my apps.
I changed every 12 apps.py files to set name = apps.myappname
INSTALLED_APPS are still defined with myappname (not apps.myappname)
Here is what I have so far:
Code works normally
Unit tests, when run app by app, 100% works for the 12 apps.
Unit tests, when run as a whole, at some point of execution, fail with errors like this one:
ImportError: Failed to import test module: apps.badges.tests
Traceback (most recent call last):
File "/usr/lib/python3.6/unittest/loader.py", line 428, in _find_test_path
module = self._get_module_from_name(name)
File "/usr/lib/python3.6/unittest/loader.py", line 369, in _get_module_from_name
__import__(name)
File "/vagrant/fugo/fugoproj/apps/badges/tests.py", line 10, in <module>
from .factories import BadgeFactory
File "/vagrant/fugo/fugoproj/apps/badges/factories.py", line 6, in <module>
from .models import Badge
File "/vagrant/fugo/fugoproj/apps/badges/models.py", line 51, in <module>
models.Model):
File "/home/ubuntu/fenv/lib/python3.6/site-packages/django/db/models/base.py", line 118, in __new__
"INSTALLED_APPS." % (module, name)
RuntimeError: Model class apps.badges.models.Badge doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.
Here is the badges/factories.py file in error:
from datetime import timedelta
import factory
from django.utils import timezone
from .models import Badge
# some code
On every test which fails, it seems related to the relative import for the model (from .models import SomeModel). If I replace this with from myappname.models import SomeModel, this solve the error.
Any idea on something wrong in my code? I'd like to keep my relative imports.
How to explain it does not fail when running tests app by app?
Thanks.
Related
https://github.com/CodeHostedHere/channel_response_bot
I want to create a db instance and share it between my two modules "messages" and "slash_commands". Extensions.py contains the code to create this and is imported in both modules. I would like to import the Flask(app) from app.py but it results in an error.
Traceback (most recent call last):
File "app.py", line 1, in <module>
from slash_commands import slash_cmds_bp
File "/opt/channel_response_bot/slash_commands/__init__.py", line 5, in <module>
from . import views
File "/opt/channel_response_bot/slash_commands/views.py", line 10, in <module>
from extensions import db, bolt_app, client, handler
File "/opt/channel_response_bot/extensions.py", line 12, in <module>
from app import flask_app
File "/opt/channel_response_bot/app.py", line 2, in <module>
from messages import messages_bp
File "/opt/channel_response_bot/messages/__init__.py", line 5, in <module>
from . import views
File "/opt/channel_response_bot/messages/views.py", line 10, in <module>
from extensions import db, bolt_app, client, handler
ImportError: cannot import name 'db' from 'extensions' (/opt/channel_response_bot/extensions.py)
I also can't do an import from __init__.py in the same level using from . import app as this results in ImportError: attempted relative import with no known parent package
I've been battling blueprints and database issues a lot so my my head is completely fried on this one, I'll need some serious ELI5 to get me across the line
Try using:
from ..extensions import db, bold_app, client, handler
My blog.models is using:
from django.contrib.auth.models import User
That model is being called from my script which I run as a task every hour. Here's the error that I'm seeing:
Traceback (most recent call last):
File "/home/redinv/src/cron.py", line 8, in <module>
from blog.models import ImagePost, MyPost, ScheduledPost
File "/home/redinv/src/blog/models.py", line 3, in <module>
from django.contrib.auth.models import User
File "/usr/lib/python3.8/site-packages/django/contrib/auth/models.py", line 2, in <module>
from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
File "/usr/lib/python3.8/site-packages/django/contrib/auth/base_user.py", line 47, in <module>
class AbstractBaseUser(models.Model):
File "/usr/lib/python3.8/site-packages/django/db/models/base.py", line 103, in __new__
app_config = apps.get_containing_app_config(module)
File "/usr/lib/python3.8/site-packages/django/apps/registry.py", line 252, in get_containing_app_config
self.check_apps_ready()
File "/usr/lib/python3.8/site-packages/django/apps/registry.py", line 134, in check_apps_ready
settings.INSTALLED_APPS
File "/usr/lib/python3.8/site-packages/django/conf/__init__.py", line 79, in __getattr__
self._setup(name)
File "/usr/lib/python3.8/site-packages/django/conf/__init__.py", line 60, in _setup
raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
2020-06-25 00:50:25 -- Completed task, took 3.95 seconds, return code was 1.
In my settings file under INSTALLED_APPS =, I already have:
'django.contrib.auth',
This code is of course working locally on my machine and I'm trying to figure what's the issue here?
I tried running that script from two locations to see if there was an issue trying to find the import files:
/home/username/src
/home/username/src/blog
Same result for both.
Any assistance is appreciate it, I'm pretty new at python.
Thanks
If you want to run Django code from outside the context of a website, you need to initialize Django first. There are two ways to do that:
Run the setup function to set things up; this part of the Django docs explains how to do that
Change the code that you want to run in your task into a management command. This is a little more work, but I would recommend it as a better option in the long term.
I have a xml parser that will store that data into a MySQL database, as part of a django app. The parser is intended to run daily, capturing some tests outputs:
`dbsync.py`:
1 #!/usr/bin/env python
2 import os
3 os.environ['DJANGO_SETTINGS_MODULE'] = 'autotester.settings'
4
5 import xml_parser
6 from models import *
The Django project is called autotester and the app is called autoreporter.
When I execute
python dbsync.py
I'm getting:
Traceback (most recent call last):
File "autoreporter/dbsync.py", line 6, in <module>
from models import *
File "/root/autotester/autoreporter/models.py", line 1, in <module>
from django.db import models
File "/usr/lib/python2.7/dist-packages/django/db/models/__init__.py", line 5, in <module>
from django.db.models.query import Q
File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line 17, in <module>
from django.db.models.deletion import Collector
File "/usr/lib/python2.7/dist-packages/django/db/models/deletion.py", line 4, in <module>
from django.db.models import signals, sql
File "/usr/lib/python2.7/dist-packages/django/db/models/sql/__init__.py", line 4, in <module>
from django.db.models.sql.subqueries import *
File "/usr/lib/python2.7/dist-packages/django/db/models/sql/subqueries.py", line 12, in <module>
from django.db.models.sql.query import Query
File "/usr/lib/python2.7/dist-packages/django/db/models/sql/query.py", line 22, in <module>
from django.db.models.sql import aggregates as base_aggregates_module
File "/usr/lib/python2.7/dist-packages/django/db/models/sql/aggregates.py", line 9, in <module>
ordinal_aggregate_field = IntegerField()
File "/usr/lib/python2.7/dist-packages/django/db/models/fields/__init__.py", line 116, in __init__
self.db_tablespace = db_tablespace or settings.DEFAULT_INDEX_TABLESPACE
File "/usr/lib/python2.7/dist-packages/django/conf/__init__.py", line 54, in __getattr__
self._setup(name)
File "/usr/lib/python2.7/dist-packages/django/conf/__init__.py", line 49, in _setup
self._wrapped = Settings(settings_module)
File "/usr/lib/python2.7/dist-packages/django/conf/__init__.py", line 132, in __init__
% (self.SETTINGS_MODULE, e)
ImportError: Could not import settings 'autotester.settings' (Is it on sys.path? Is there an import error in the settings file?): No module named autotester.settings
How can I have DJANGO_SETTINGS_MODULE set, so I can execute the dbsync.py script properly?
Two issues
You need to import models from your app, so provided that your Django project is in the PYTHONPATH you can simply import the app models by:
from autoreporter.models import *
or do a relative import, if dbsync.py(looks from your traceback) is in the app directory.
from .models import *
Add Django project to PYTHONPATH. You need to have the Django project in the PYTHONPATH to it be accessed from your dbsync.py(provided it is in your django app directory), quick easy fix for this would be to do the following in your code to dbsync.py.
import sys
import os
##get project directory
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
##get project parent directory and add to python path using sys.path.append
SYS_PATH = os.path.dirname(BASE_DIR)
if SYS_PATH not in sys.path:
sys.path.append(SYS_PATH)
os.environ['DJANGO_SETTINGS_MODULE'] = 'autotester.settings'
I run a few scripts using the ORM without the rest of Django. I wouldn't set the environment in the script.
Since Django 1.7 you need to import your models, and run django setup.
I would set the environment as you do for the main Django application. Then at the start of your script add these lines. Those should set up the Django environment so it can access your models the same way a standard Django app will.
import django
django.setup()
import xml_parser
from .models import *
I want to run my first Django cassandra code in pyCharm.
My code is running smoothly in Django console but it's not working in a .py file. these are the errors:
C:\Python27\python.exe "D:/Developer Center/PyCharm/DJangoCassandra/MyApp/testFile.py"
Traceback (most recent call last):
File "D:/Developer Center/PyCharm/DJangoCassandra/MyApp/testFile.py", line 3, in <module>
from MyApp.models import Person
File "D:\Developer Center\PyCharm\DJangoCassandra\MyApp\models.py", line 1, in <module>
from django.db import models
File "C:\Python27\lib\site-packages\django\db\__init__.py", line 11, in <module>
if settings.DATABASES and DEFAULT_DB_ALIAS not in settings.DATABASES:
File "C:\Python27\lib\site-packages\django\conf\__init__.py", line 53, in __getattr__
self._setup(name)
File "C:\Python27\lib\site-packages\django\conf\__init__.py", line 46, in _setup
% (desc, ENVIRONMENT_VARIABLE))
django.core.exceptions.ImproperlyConfigured: Requested setting DATABASES, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
Process finished with exit code 1
this is my .py file:
__author__ = 'ehsan'
from cqlengine import connection
from MyApp.models import Person
from cqlengine.management import sync_table,drop_table
def main():
connection.setup(['127.0.0.1:9160'])
sync_table(Person)
Person.create(id='2',name='Ali',family='Rezayee')
p = Person.objects.all()
for item in p:
print item.id
Person.filter(id='1')
this is my model:
from django.db import models
from cqlengine import Model, columns
# Create your models here.
class Person(Model):
id = columns.Text(primary_key=True)
name = columns.Text()
family = columns.Text()
If you'll write the following lines when starting the managy.py shell command, it should work:
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "zoo.settings")
I'm having a module import issue.
using python 2.6 on ubuntu 10.10
I have a class that subclasses the daemon at http://www.jejik.com/articles/2007/02/a_simple_unix_linux_daemon_in_python/ . I created a python package with a module containing code that imports some models from a django project. The code works when used from a class, not subclassing the daemon. The structure looks something like:
my_module
__init__.py
- bin
- cfg
- core
__init__.py
collection.py
daemon.py
The ItemRepository code:
class ItemRepository(object):
"""This class provides an implementation to retrieve configuration data
from the monocle database
"""
def __init__(self, project_path):
if project_path is not None:
sys.path.append(project_path)
try:
from django.core.management import setup_environ
from someproj import settings
setup_environ(settings)
from someproj.someapp.models import ItemConfiguration
except ImportError:
print "Could not import models from web app. Please ensure the\
PYTHONPATH is configured properly"
def get_scheduled_collectors(self):
"""This method finds and returns all ItemConfiguration objects
that are scheduled to run
"""
logging.info('At the error path: %s' % sys.path)
# query ItemConfigs from database
items = ItemConfiguration.objects.filter(disabled=False) #ERROR OCCURS AT THIS LINE
return [item for item in items if item.scheduled]
The daemon code (in /usr/local/bin/testdaemon.py):
import sys
from my_module.core.daemon import Daemon
from my_module.core.collection import ItemRepository
import logging
import time
class TestDaemon(Daemon):
default_conf = '/etc/echodaemon.conf'
section = 'echo'
def run(self):
while True:
logging.info('The echo daemon says hello')
ir = ItemRepository(project_path=self.project_path)
items = ir.get_scheduled_collectors() #TRIGGERS ERROR
logging.info('there are %d scheduled items' % len(items))
time.sleep(1)
if __name__ == '__main__':
TestDaemon().main()
The error I get is "NameError: global name 'my_module' is not defined" It get's past the import but then fails when trying to call a method on the object. I'm assuming it has to do with sys.path / PYTHONPATH, but I know my django project is on the path, as I've printed it out. Nothing so far in the python docs or Learning Python has helped yet. Does anyone have any insights or know of a good reference to module imports?
UPDATE:
Now I've attempted to simplify the problem to make it easier to understand. Now I have a directory structure that looks like:
/home
/username
/django
/someproj
/web
models.py
/my_module
daemon.py
I have set the $PYTHONPATH variable in /etc/bash.bashrc to '/home/username/django'.
Inside the testdaemon.py file the imports look like:
import logging
from django.core.management import setup_environ
from someproj import settings
setup_environ(settings)
from someproj.web.models import ItemConfiguration
But now I get an ImportError: No module named 'someproj'. So then I appended the path.
import sys
sys.path.append('/home/username/django')
import logging
from django.core.management import setup_environ
from someproj import settings
setup_environ(settings)
from someproj.web.models import ItemConfiguration
And now the ImportError says: No module named 'web'. Here's the traceback:
Traceback (most recent call last):
File "testdaemon.py", line 77, in <module>
TestDaemon('/tmp/testdaemon.pid').run()
File "testdaemon.py", line 47, in run
scheduled_items = [item for item in ItemConfiguration.objects.filter(disabled=False) if collector.scheduled]
File "/usr/local/lib/python2.6/dist-packages/django/db/models/manager.py", line 141, in filter
return self.get_query_set().filter(*args, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 550, in filter
return self._filter_or_exclude(False, *args, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 568, in _filter_or_exclude
clone.query.add_q(Q(*args, **kwargs))
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/query.py", line 1128, in add_q
can_reuse=used_aliases)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/query.py", line 1026, in add_filter
negate=negate, process_extras=process_extras)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/query.py", line 1179, in setup_joins
field, model, direct, m2m = opts.get_field_by_name(name)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/options.py", line 291, in get_field_by_name
cache = self.init_name_map()
File "/usr/local/lib/python2.6/dist-packages/django/db/models/options.py", line 321, in init_name_map
for f, model in self.get_all_related_m2m_objects_with_model():
File "/usr/local/lib/python2.6/dist-packages/django/db/models/options.py", line 396, in get_all_related_m2m_objects_with_model
cache = self._fill_related_many_to_many_cache()
File "/usr/local/lib/python2.6/dist-packages/django/db/models/options.py", line 410, in _fill_related_many_to_many_cache
for klass in get_models():
File "/usr/local/lib/python2.6/dist-packages/django/db/models/loading.py", line 167, in get_models
self._populate()
File "/usr/local/lib/python2.6/dist-packages/django/db/models/loading.py", line 61, in _populate
self.load_app(app_name, True)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/loading.py", line 76, in load_app
app_module = import_module(app_name)
File "/usr/local/lib/python2.6/dist-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
ImportError: No module named web
So going from the earlier comments I tried adding:
import sys
sys.path.append('/home/username/django')
import logging
from django.core.management import setup_environ
from someproj import settings
setup_environ(settings)
from someproj import web
from someproj.web import models
from someproj.web.models import ItemConfiguration
But that didn't help. So I created a very simple file:
#!/usr/bin/python
import logging
import time
import sys
sys.path.append('/home/username/django')
from django.core.management import setup_environ
from someproj import settings
setup_environ(settings)
from someproj.web.models import ItemConfiguration
if __name__ == '__main__':
print sys.path
items = ItemConfiguration.objects.all()
for item in items:
print item
And this works! Which really only further confuses me. So now I'm thinking maybe it has to do with the daemon. It uses os.fork() and I'm not sure if the path is still set. This is why I set the $PYTHONPATH variable in the /etc/bash.bashrc file.
Any more insights? I really need the daemon, I don't have much of a choice as I need a long running process.
With from my_module.core.daemon import Daemon you do not actually bind the loaded module my_module to a variable. Use import my_module
just before or after your other imports to do that.
Explained in code:
>>> from xml import dom
>>> xml
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'xml' is not defined
>>> import xml
>>> xml
<module 'xml' from '/usr/lib/python2.6/xml/__init__.pyc'>
It ended up being that I needed to reference the fully qualified name of my app in my Django project's settings.py file in the INSTALLED_APPS setting. "someproj.web" instead of just "web". Using the shorter version works fine within a Django project, just not so well from the outside.
you get me confused what do you mean "your Django project" is your module "my_module" is part of an higher package? something like this
django_project
|
|my_module
__init__.py
- bin
- cfg
- core
__init__.py
collection.py
daemon.py
if so , and if like you said django_project is in PYTHONPATH , so you should just import my_module like this:
from django_project.my_module.core.daemon import Daemon
by the way it's preferable to import module no class neither function like this:
from django_project.my_module.core import daemon
and use it like this:
daemon.Daemon()