When i run the same code in django shell it works fine for me. but when I fire up the Python interpreter (Python 2) to check some things, I get the an error when I try importing - from django.contrib.auth.models import User
import os
import django
django.setup()
import smtplib
import sys
sys.path.insert(0, "/edx/app/edxapp/edx-platform/lms/djangoapps/courseware")
import dateutil.relativedelta
from django.conf import settings
from django.contrib.auth.models import User
from opaque_keys.edx.keys import CourseKey
from courseware.models import StudentModule
from datetime import datetime
def PCSurvey_email():
for student in StudentModule.objects.filter(module_type="PCSurvey"):
two_months_date = datetime.now().date() - dateutil.relativedelta.relativedelta(months=2)
created = student.created.date()
if created == two_months_date :
user_id = student.student_id
user_email = User.objects.get(id = user_id).email
FROM = "user#example.com"
TO = [user_email] # must be a list
SUBJECT = "Hello!"
TEXT = "This message was sent with Python's smtplib."
# Prepare actual message
message = """\
From: %s
To: %s
Subject: %s
%s
""" % (FROM, ", ".join(TO), SUBJECT, TEXT)
# Send the mail
server = smtplib.SMTP('outlook.net')
server.sendmail(FROM, TO, message)
server.quit()
#deleting the module
smod = StudentModule.objects.get(module_type="PCSurvey", course_id = student.course_id, student_id= student.student_id)
smod.delete()
The error that I get is
Traceback (most recent call last):
File "/edx/app/edxapp/edx-platform/lms/djangoapps/courseware/PCSurvey_email.py", line 4, in <module>
django.setup()
File "/usr/local/lib/python2.7/dist-packages/django/__init__.py", line 22, in setup
configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 53, in __getattr__
self._setup(name)
File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 39, in _setup
% (desc, ENVIRONMENT_VARIABLE))
django.core.exceptions.ImproperlyConfigured: Requested setting LOGGING_CONFIG, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
Any help on this is very much appreciated.
Django needs to be setup, in order to run fine, when you run it through manage.py shell, manage.py takes care or setting it up for you, but doing it via a python script, needs manual setup.
You also seem to have used your defined models, to be able to import them into your python script, you need to add the path to the root folder of your project to the current python path.
Finally, you need to tell django where your settings file is (before setting up your django), in your manage.py file, you should have something like this :
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myapp.settings")
Go make it a constant, name it * DEFAULT_SETTINGS_MODULE *, so you now have:
os.environ.setdefault("DJANGO_SETTINGS_MODULE", DEFAULT_SETTINGS_MODULE)
Now you need to import the constant into your script and tell django (By setting an env var) where it should look for the settings file.
So in all:
import sys, os
sys.path.insert(0, "/path/to/parent/of/courseware") # /home/projects/my-djproj
from manage import DEFAULT_SETTINGS_MODULE
os.environ.setdefault("DJANGO_SETTINGS_MODULE", DEFAULT_SETTINGS_MODULE)
import django
django.setup()
This way you're setup up just fine.
For new users coming on this question.
If you don't want to setup django settings file manually as described wonderfully by #SpiXel
You can run python manage.py shell from the project directory which will basically link the settings file that django needs automatically.
When you open a shell, django automatically sets itself up.
With Python interpreter you have to do this manually:
import django
django.setup()
Just a simple solution all can try!
I tried to run my custom script inside shell only by using the following way:
python manage.py shell < /path_to_script
Related
I'm having difficulty calling a django script. Here is what I'm currently doing in my root directory:
>>> import os
>>> os.environ['DJANGO_SETTINGS_MODULE'] = 'settings.py'
>>> from django.conf import settings
>>> settings.configure()
>>> settings.DATABASES
{}
The settings.DATABASES should not be empty, so I know I haven't initialized the project correct. How would I do this in django2.1? I used to be able to do this easily using import settings; setup_environ(settings), but not anymore.
Note: I'm looking to be able to run the above from any directory. Here is an example from trying to import my project from tmp:
(V) david$ cd /tmp && python
>>> import django
>>> from django.conf import settings
>>> settings.configure()
>>> django.setup()
>>> from users.models import *
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'users'
The 2.1 docs state that you need to use setup() now. Relevant snippet:
import django
from django.conf import settings
settings.configure()
django.setup()
# Now this script or any imported module can use any part of Django it needs.
from myapp import models
Here is the full documentation.
You don't need to run settings.configure(). To properly initialize, you could do something like:
if __name__ == '__main__':
import sys, os
sys.path.append(django_root)
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
from django.conf import settings
print (settings.DATABASES)
I have had this problem and my solution (script) is the result of hundreds of Django sites over the course of a decade.
Some solutions can inhibit the use of settings and this is the best working version of a stand alone script for running Django that I have been able to compile. This is faster than the Django shell generally (no need to exit and re-enter).
Above suggestions will work 98% of the time. Still, consider reviewing (and commenting) on this for a more robust way to run Django scripts. I have hit the "Exceptions" about 1-2x every year for a while until I got this script fully debugged via many projects.
"""
scratch.py
----->note the no .py-----v
clear; echo 'import scratch' | python manage.py shell
"""
from django.conf import settings
from django.core.wsgi import get_wsgi_application # turns on app, access to DB
print(f'- - - - - - - - - - - - - - - - - - - - ')
print(f'settings: {settings}')
print('settings.DEBUG: {}'.format(settings.DEBUG))
# settings.DEBUG = True
# etc
if not settings.DEBUG:
# keeps this off prod, usually on git ignore as well
print('SETTINGS NOT IN DEBUG, set to:')
print(settings.DEBUG)
raise(RuntimeError('Can not be run on production or when debug is False'))
application = get_wsgi_application()
print('READY!')
print(f'- - - - - - - - - - - - - - - - - - - - ')
# App ready, code below. Add imports, calls, etc below here.
Additionally this script can be run in an infinite loop, such as for monitoring, debugging, and other day to day Django operations. At the ned of the file just add the following, possibly with a sleep() and logging.
while True:
pass
Call this script scratch.py from the same directory as manage.py. Call using the following syntax:
echo 'import scratch' | python manage.py shell
If you change the file name, then the import will need to be edited. This syntax loads the settings without needing to call settings.configure(). We leverage manage.py and that seems to solve the edge cases.
Note: This will NOT auto-reload on with changes.
I have the following script and it's meant to be a standalone Django script so I can run python my_script.py from the command line. It used to work with Django 1.8, after I upgrade to Django 1.11, I'm getting the following error:
Traceback (most recent call last):
File "app.py", line 8, in <module>
django.setup()
File "C:\python27\lib\site-packages\django-1.11.5-py2.7.egg\django\__init__.py", line 22, in setup
configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
File "C:\python27\lib\site-packages\django-1.11.5-py2.7.egg\django\conf\__init__.py", line 56, in __getattr__
self._setup(name)
File "C:\python27\lib\site-packages\django-1.11.5-py2.7.egg\django\conf\__init__.py", line 41, in _setup
self._wrapped = Settings(settings_module)
File "C:\python27\lib\site-packages\django-1.11.5-py2.7.egg\django\conf\__init__.py", line 110, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
File "C:\python27\lib\importlib\__init__.py", line 37, in import_module
__import__(name)
ImportError: Import by filename is not supported.
This is my python script
# standalone django setup
import os, sys, logging, django
prj_dir = os.path.abspath(os.path.dirname(os.path.dirname(__file__)))
logging.basicConfig(level=logging.INFO)
logging.info("PRJ_DIR: %s" % prj_dir)
sys.path.append(prj_dir)
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "%s.settings" % prj_dir.split("/")[-1])
django.setup()
...
...
It seems you're trying to split a file path by the forward slash / while running your script on Windows where path separator is the backslash \. Use os.path.basename instead of manually dealing with the stings (.split("/")[-1]):
>>> import os
>>> os.path.basename(r'/home/user/project')
'project'
>>> os.path.basename(r'c:\users\user\project')
'project'
In comparison:
>>> r'/home/user/project'.split("/")[-1]
'project' # works
>>> r'c:\users\user\project'.split("/")[-1]
'c:\\users\\user\\project' # error
Don't reinvent the wheel. There is a cool manage command on django extensions that lets you run scripts on the django context.
https://django-extensions.readthedocs.io/en/latest/runscript.html
Others have pointed out an issue with your path manipulation, however the cause of your actual import exception is that you are setting DJANGO_SETTINGS_MODULE to the name of a file.
It needs to be the name of a Python module. In other words, your settings file should end in .py and you should pass the name of it (without .py), or the importable path to it - exactly as you would type it in an import statement.
If your settings are in my_settings.py, then you would set the variable to my_settings.
Here is a simplified example of a one-page django application taken from the excellent Lightweight Django book's excerpt:
import sys
from django.conf import settings
settings.configure(
DEBUG=True,
SECRET_KEY='thisisthesecretkey',
ROOT_URLCONF=__name__,
MIDDLEWARE_CLASSES=(
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
),
)
from django.conf.urls import url
from django.http import HttpResponse
def index(request):
return HttpResponse('Hello World')
urlpatterns = (
url(r'^$', index),
)
if __name__ == "__main__":
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
The error is in
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "%s.settings" % prj_dir.split("/")[-1])
If you run this script from the same folder as manage.py, it will return the name of the manage.py folder instead settings.py folder resulting managepy_folder.settings
when it should output settingspy_folder.settings to access the app
Test with :
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settingspy_folder.settings")
if it that was the problem, if you want use the same algorithm or something similar in multiple projects, you need a pattern to modify the current folder name to indicate the settings.py folders name, using the same folders name in both wouldn't require modification.
if that doesn't fix please post your settings.py file
If your script is in the same folder as settings.py, the code should be like this:
import os, sys, logging, django
prj_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
logging.basicConfig(level=logging.INFO)
logging.info("PRJ_DIR: %s" % prj_dir)
sys.path.append(prj_dir)
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "{}.settings".format(os.path.basename(prj_dir)))
django.setup()
else, if your script is in the parent folder of settings.py:
import os, sys, logging, django
prj_dir = os.path.dirname(os.path.abspath(__file__))
logging.basicConfig(level=logging.INFO)
logging.info("PRJ_DIR: %s" % prj_dir)
sys.path.append(prj_dir)
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "{}.settings".format(os.path.basename(prj_dir)))
django.setup()
in the line:
os.environ.setdefault(
"DJANGO_SETTINGS_MODULE",
"myproject.settings") # should be like in import command
)
Just use python module path: like "myproject.settings" instead of path like /opt/myproject/settings" or "c:\myproject\settings.py"
I'm writing a program that enables my Django application to consume a REST API. My problem is that Django rejects the program if I run it as a .py file, but accepts it via shell. The program is divided into two files: one that interacts with the API, and another that interacts with Django. The main folder structure is as follows:
/root folder
[rest of Django project]
-__init__.py
-test.py
-mid_side.py
-teamup_api/
-__init__.py
-teamup.py
where test.py is the file that starts the program, mid_side.py is the Django-side part, and teamup.py is the API-side part. Below is the specific code I'm having trouble with.
test.py
import django
from mid_side import dates
from datetime import datetime
start = datetime(2017,7,28)
end = datetime(2017,8,19)
test = dates(start,end)
print(test)
Where dates is a method from mid_side that accepts two datetime objects as parameters.
According to Django, the problem emerges in mid_side.py, as it imports models from an app in the main Django project. The specific line Django has issues with is from events.models import Service, where events is an app Django uses and Services is a model. The traceback is
Traceback (most recent call last):
File "test.py", line 2, in <module>
from mid_side import dates
File "/home/brian/Github/nydkc11/nydkcd11/mid_side.py", line 3, in <module>
from events.models import Service
File "/home/brian/Github/nydkc11/nydkcd11/events/models.py", line 1, in <module>
from embed_video.fields import EmbedVideoField
File "/home/brian/anaconda3/envs/server/lib/python3.6/site-packages/embed_video/fields.py", line 6, in <module>
from .backends import detect_backend, UnknownIdException, \
File "/home/brian/anaconda3/envs/server/lib/python3.6/site-packages/embed_video/backends.py", line 17, in <module>
from .settings import EMBED_VIDEO_BACKENDS, EMBED_VIDEO_TIMEOUT, \
File "/home/brian/anaconda3/envs/server/lib/python3.6/site-packages/embed_video/settings.py", line 7, in <module>
'embed_video.backends.SoundCloudBackend',
File "/home/brian/anaconda3/envs/server/lib/python3.6/site-packages/django/conf/__init__.py", line 56, in __getattr__
self._setup(name)
File "/home/brian/anaconda3/envs/server/lib/python3.6/site-packages/django/conf/__init__.py", line 39, in _setup
% (desc, ENVIRONMENT_VARIABLE))
django.core.exceptions.ImproperlyConfigured: Requested setting EMBED_VIDEO_BACKENDS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
I know for a fact the traceback concerns an app my project uses, django-embed-video. What I don't understand is why Django is indicating it as a problem considering a) Service doesn't use fields from django-embed-video, b) I successfully deployed the application on my project earlier. My questions are thus as follows:
1) Why does this problem occur in the .py file and not in shell?
2) What is causing Django to output the traceback?
Below is relevant code from the program. Please notify me if additional clarifications are required. Thank you!
*I decided not to include teamup.py as this is more of a Django-side problem, but I will include it if requested.
mid_side.py
from dateutil.parser import parse
from events.models import Service
import datetime
from teamup_api.teamup import *
def dates(start,end):
event_list = day_query(start.year, start.month, start.day, end.year, end.month, end.day)['events']
service_objects = []
for event in event_list:
service_objects.append(Service(
title=event['title'],
school=none_parse(event['who']),
location=none_parse(event['location']),
start_time=parse(event['start_dt']),
end_time=parse(event['end_dt']),
all_day=event['all_day'],
description=none_parse(event['notes'])
))
return service_objects
models.py
class Service(models.Model):
title = models.CharField(max_length=100)
school = models.CharField(max_length=100)
location = models.CharField(max_length=100)
start_time = models.DateTimeField()
end_time = models.DateTimeField()
all_day = models.BooleanField()
description = models.TextField()
def __str__(self):
return self.title
UPDATE: I've been running 'shell' through manage.py shell, and 'console' via natively running a python file (e.g. python [file].py)
My problem was probably that I was using Django resources in the native python environment. As Chris indicates in the comments, creating a custom manage.py command instead of a .py start file should suffice.
I've had a similar problem to this one before where I received a nearly identical error message, i.e.
C:\Users\David J\optilab\optilab_project>python functional_tests.py
Traceback (most recent call last):
File "functional_tests.py", line 4, in <module>
from optilab.models import Design
File "C:\Users\David J\optilab\optilab_project\optilab\models.py", line 10, in <module>
class Design(models.Model):
File "C:\Users\David J\optilab\optilab_project\optilab\models.py", line 11, in Design
name = models.CharField(max_length=30)
File "C:\Python27\lib\site-packages\django\db\models\fields\__init__.py", line 1072, in __init__
super(CharField, self).__init__(*args, **kwargs)
File "C:\Python27\lib\site-packages\django\db\models\fields\__init__.py", line 166, in __init__
self.db_tablespace = db_tablespace or settings.DEFAULT_INDEX_TABLESPACE
File "C:\Python27\lib\site-packages\django\conf\__init__.py", line 55, in __getattr__
self._setup(name)
File "C:\Python27\lib\site-packages\django\conf\__init__.py", line 41, in _setup
% (desc, ENVIRONMENT_VARIABLE))
django.core.exceptions.ImproperlyConfigured: Requested setting DEFAULT_INDEX_TABLESPACE, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
But it was being produced because I was calling my test incorrectly from the shell. Now I am attempting to run a functional test, which I have done several times before successfully using the command
"python functional_tests.py"
in my project folder -- but it produces the above error message. Can anyone tell me what I'm doing wrong, or why this may have worked before but not now? thank.
Edit:
Here is functional_tests.py:
from selenium import webdriver
import unittest
from selenium.webdriver.common.keys import Keys
from optilab.models import Design
from automa.api import *
class HomeTest(unittest.TestCase):
def setUp(self):
self.browser = webdriver.Firefox()
self.browser.get('http://localhost:8000/lab')
self.new_design = self.browser.find_element_by_id("new_design")
self.load_design = self.browser.find_element_by_id("load_test")
self.design_name = self.browser.find_element_by_id("design_name")
self.browser.implicitly_wait(3)
def test_latest_designs_displayed_in_table(self):
#
self.browser.implicitly_wait(3)
self.design_table = self.browser.find_element_by_id("design_table")
self.rows = self.browser.find_elements_by_tag_name('tr')
self.assertTrue(
any(row.text == 'My First Design' for row in rows),
"New design did not appear in test table."
)
def test_entering_text_activates_new(self):
# user enters name of design. new becomes clickable.
self.browser.implicitly_wait(3)
self.design_name.send_keys("sample_design")
def test_new_test_can_be_saved_and_retrieved(self):
name = "sample design"
self.design_name.send_keys(name)
self.new_design.click()
self.browser.implicitly_wait(3)
self.saved_design = Design.objects.get(name=design_name)
def test_entering_name_of_previously_saved_test_activates_load(self):
# user enters the name of a previously saved test. load button becomes clickable.
design = Design(name="sample_design")
saved_design = Design.objects.get(id=1).name
self.browser.implicitly_wait(3)
self.design_name.send_keys(saved_design)
self.load_design.click()
def test_entering_name_of_previously_saved_test_but_clicking_new_prompts_overwrite(self):
# user entes name of saved test but clicks new. dialogue box asks if test should be overwritten.
previously_saved_design = Design.objects.get(id=1).name
self.design_name.send_keys(previously_saved_design)
def test_clicking_save_redirects_to_model(self):
pass
def tearDown(self):
self.browser.quit()
if __name__ == '__main__':
unittest.main()
The start of your script should look something like this
import os, sys
sys.path.append(os.getcwd())
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "optilab.settings") # or whatever
import django
django.setup()
from selenium import webdriver
import unittest
from selenium.webdriver.common.keys import Keys
from optilab.models import Design
from automa.api import * # try not to do this
This happens because Django has not been correctly setup before trying to call settings, it has no way to know which project settings should be used without the "DJANGO_SETTINGS_MODULE" variable and the django.setup() call is essential to import models and access the database. (see e4c5's answer for that method)
However, this is not the recommended method or the easiest one. You can call the functional tests just like the unit tests from manage.py test and Django will automagically setup everything for you, it'll even use a tempdb for functional tests. If your functional tests include insertions and updates in the functional tests (and you should) a tempdb comes in handy.
To use this facility though, you must change a few things in your code.
First there is the matter of the filename, the default pattern for manage.py test(doc) is "test*.py", you can change this with -p or --pattern but the usual solution is to rename functional_tests.py to tests.py and place it inside a functional_tests folder. That way you can call just the functional tests by calling manage.py test functional_tests.
Also, instead of importing TestCase from unittest you should look into using TestCase(doc) from django.test or even LiveServerTestCase(doc) which would save you the trouble of running manage.py runserver yourself.
I'm having a problem with Django. If you could help me, I'd very much appreciate it.
So, I've found lots of people with a similar problem, but none of the "solutions" work for me. I'm learning how to use django and I've successfully installed it.
But, I get an error when executing the import from the init.py file:
from django.db import connections
Here's the error:
raise ImportError("Could not import settings '%s' (Is it on sys.path?): %s" % (self.SETTINGS_MODULE, e))
ImportError: Could not import settings 'TestProject.settings' (Is it on sys.path?): cannot import name connections
The containing django folder is included on my PYTHONPATH, and my file structure looks like this:
TestProject
--manage.py
--TestProject
----__init__.py
----settings.py
etc.
This is my manage.py file:
#!/usr/bin/env python import os import sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "TestProject.settings")
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
I don't really know how to troubleshoot this. I'm also new to Python, so forgive me if this is completely obvious to someone else.
Thanks for your help.
This is a weird quirk indeed. I found the following to work in my program:
Import django settings.
Set the default environment.
ACCESS e.g. SITE_ID (or another property, but just use e.g. SITE_ID) through settings.
Finally, you should be able to import.
E.g.
from django.conf import settings
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myapp.settings")
settings.SITE_ID
or use another variable like DEBUG; note.. this line does nothing.. it just accesses the settings and seems to 'activate' the environment. You could also do a simple print statement like print settings.SITE_ID, but that's not necessary.)
from myapp.models import MyModel, MyOtherModel
If you do everything in that order.. well it worked for me! The magic happens when you access an actual setting before you do the import of a model.
I think this might be a bug/error in Django 1.5